2024-09-11T09:17:57.253 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T09:17:57.269 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T09:17:57.347 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900713' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: extra_packages: - rbd-nbd flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: daaf78de334647fc0c800f4ac495cc9475a26bae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 6175 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 88/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: daaf78de334647fc0c800f4ac495cc9475a26bae targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKUc2QfOB6XuJyv1zZxWvtUzcJDsvpb2kFQwcW/1jYYrAgPUaaHFiSWNknMxowsuXgN+kdXDiGwm1Ow73CRekU= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMF7fhZ/UN5Ejb5s5dMmizz+6O3picx+MvgVyUGKkvLzIs4g0Xe2vd+hUkcDXqkgjKFINOYDMrQGf09vRwrNq/E= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAhRXNKiFBS44HYqCY9UeX03Sa8Cfgpd/kISiOWbfEQrVI7XQKtdJkhmyzDaFLjNXOEYcjPNOxNpr2IZ+XY3Kk= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_21:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T09:17:57.348 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa; will attempt to use it 2024-09-11T09:17:57.348 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa/tasks 2024-09-11T09:17:57.349 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T09:17:57.350 INFO:teuthology.task.internal:Checking packages... 2024-09-11T09:17:57.371 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T09:17:57.371 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T09:17:57.371 INFO:teuthology.packaging:ref: None 2024-09-11T09:17:57.371 INFO:teuthology.packaging:tag: None 2024-09-11T09:17:57.371 INFO:teuthology.packaging:branch: squid 2024-09-11T09:17:57.371 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:17:57.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T09:17:57.604 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-308.gdaaf78de 2024-09-11T09:17:57.606 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T09:17:57.615 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T09:17:57.615 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T09:17:57.619 INFO:teuthology.task.internal:Saving configuration 2024-09-11T09:17:57.632 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T09:17:57.636 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T09:17:57.659 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 09:12:59.472473', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKUc2QfOB6XuJyv1zZxWvtUzcJDsvpb2kFQwcW/1jYYrAgPUaaHFiSWNknMxowsuXgN+kdXDiGwm1Ow73CRekU='} 2024-09-11T09:17:57.679 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 09:12:59.470590', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMF7fhZ/UN5Ejb5s5dMmizz+6O3picx+MvgVyUGKkvLzIs4g0Xe2vd+hUkcDXqkgjKFINOYDMrQGf09vRwrNq/E='} 2024-09-11T09:17:57.701 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 09:12:59.474108', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAhRXNKiFBS44HYqCY9UeX03Sa8Cfgpd/kISiOWbfEQrVI7XQKtdJkhmyzDaFLjNXOEYcjPNOxNpr2IZ+XY3Kk='} 2024-09-11T09:17:57.702 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T09:17:57.707 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-11T09:17:57.707 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-11T09:17:57.708 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['client.0'] 2024-09-11T09:17:57.708 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T09:17:57.792 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5019cf3ac0>, signals=[15]) 2024-09-11T09:17:57.792 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T09:17:57.798 INFO:teuthology.task.internal:Opening connections... 2024-09-11T09:17:57.798 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:17:57.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:17:57.875 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:17:57.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:17:57.947 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-11T09:17:57.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:17:58.021 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T09:17:58.027 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-09-11T09:17:58.046 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-09-11T09:17:58.046 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-09-11T09:17:58.103 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="CentOS Stream" 2024-09-11T09:17:58.103 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="9" 2024-09-11T09:17:58.103 INFO:teuthology.orchestra.run.smithi017.stdout:ID="centos" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="rhel fedora" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="9" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el9" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:LOGO="fedora-logo-icon" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://centos.org/" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T09:17:58.104 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T09:17:58.105 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T09:17:58.105 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-09-11T09:17:58.133 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-11T09:17:58.151 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-11T09:17:58.151 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-11T09:17:58.209 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-09-11T09:17:58.209 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-09-11T09:17:58.209 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-09-11T09:17:58.209 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T09:17:58.210 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T09:17:58.211 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-11T09:17:58.233 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-11T09:17:58.251 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-11T09:17:58.251 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-11T09:17:58.308 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-11T09:17:58.308 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-11T09:17:58.308 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-11T09:17:58.308 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-11T09:17:58.308 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T09:17:58.309 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T09:17:58.309 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-11T09:17:58.341 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T09:17:58.358 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T09:17:58.364 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T09:17:58.364 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-09-11T09:17:58.366 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-11T09:17:58.368 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-11T09:17:58.385 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T09:17:58.391 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T09:17:58.391 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-09-11T09:17:58.424 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-11T09:17:58.426 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-11T09:17:58.534 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T09:17:58.592 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T09:17:58.606 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T09:17:58.606 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T09:17:58.606 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T09:17:58.608 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T09:17:58.608 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T09:17:58.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:17:58.625 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-11T09:17:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:17:58.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:17:58.627 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-11T09:17:58.627 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-09-11T09:17:58.682 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-503.el9.x86_64 2024-09-11T09:17:58.682 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-503.el9.x86_64 2024-09-11T09:17:58.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-09-11T09:17:58.684 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-503.el9.x86_64 2024-09-11T09:17:58.684 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-503.el9.x86_64 2024-09-11T09:17:58.684 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-09-11T09:17:58.685 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-503.el9.x86_64 2024-09-11T09:17:58.685 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.14.0-503.el9.x86_64 2024-09-11T09:17:58.685 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum install -y kernel 2024-09-11T09:18:00.944 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-11T09:18:01.965 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.2 MB 00:01 2024-09-11T09:18:02.323 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-11T09:18:03.982 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-11T09:18:04.154 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 20 MB 00:02 2024-09-11T09:18:05.590 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 20 MB 00:02 2024-09-11T09:18:08.529 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-11T09:18:09.749 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-11T09:18:09.971 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-11T09:18:10.207 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-11T09:18:11.067 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-11T09:18:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-11T09:18:11.633 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-11T09:18:12.078 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T09:18:12.537 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T09:18:16.681 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T09:18:17.500 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-11T09:18:18.026 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T09:18:18.763 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T09:18:18.763 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T09:18:18.873 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:18:18.884 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:18:18.885 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:18:18.885 INFO:teuthology.orchestra.run.smithi017.stdout:Install 4 Packages 2024-09-11T09:18:18.885 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:18:18.885 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 79 M 2024-09-11T09:18:18.885 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 126 M 2024-09-11T09:18:18.885 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:18:19.116 INFO:teuthology.orchestra.run.smithi017.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 199 kB/s | 23 kB 00:00 2024-09-11T09:18:19.507 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T09:18:19.507 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T09:18:19.591 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:18:19.596 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:18:19.596 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-11T09:18:19.596 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:18:19.596 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-11T09:18:19.597 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:18:19.598 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 79 M 2024-09-11T09:18:19.598 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 126 M 2024-09-11T09:18:19.598 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:18:19.650 INFO:teuthology.orchestra.run.smithi017.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-11T09:18:19.966 INFO:teuthology.orchestra.run.smithi017.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-11T09:18:20.065 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T09:18:20.065 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T09:18:20.146 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T09:18:20.157 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T09:18:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:18:20.158 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:18:20.158 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:18:20.158 INFO:teuthology.orchestra.run.smithi096.stdout:Install 4 Packages 2024-09-11T09:18:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:18:20.158 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 79 M 2024-09-11T09:18:20.159 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 126 M 2024-09-11T09:18:20.159 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:18:20.300 INFO:teuthology.orchestra.run.smithi017.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-11T09:18:20.300 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:18:20.300 INFO:teuthology.orchestra.run.smithi017.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-11T09:18:20.368 INFO:teuthology.orchestra.run.smithi096.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 252 kB/s | 23 kB 00:00 2024-09-11T09:18:20.435 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 39 kB/s | 23 kB 00:00 2024-09-11T09:18:20.744 INFO:teuthology.orchestra.run.smithi096.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-11T09:18:21.015 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:18:21.136 INFO:teuthology.orchestra.run.smithi096.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 40 MB/s | 35 MB 00:00 2024-09-11T09:18:21.380 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:18:21.380 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:18:21.544 INFO:teuthology.orchestra.run.smithi096.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-11T09:18:21.545 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:18:21.545 INFO:teuthology.orchestra.run.smithi096.stdout:Total 57 MB/s | 79 MB 00:01 2024-09-11T09:18:21.860 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.8 MB/s | 16 MB 00:02 2024-09-11T09:18:22.230 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:18:22.387 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:18:22.388 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:18:22.603 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:18:22.603 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:18:23.583 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:18:23.583 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:18:24.502 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.4 MB/s | 35 MB 00:04 2024-09-11T09:18:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:18:25.029 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.2 MB/s | 29 MB 00:04 2024-09-11T09:18:25.029 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:18:25.029 INFO:teuthology.orchestra.run.smithi107.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-11T09:18:25.628 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T09:18:25.710 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:18:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:18:26.044 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:18:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:18:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:18:26.706 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T09:18:26.787 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:18:27.011 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:18:27.011 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:18:28.510 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:18:28.636 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:18:29.471 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:18:29.574 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:18:29.697 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:18:30.120 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T09:18:30.219 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:18:32.999 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:18:33.129 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:18:33.756 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:18:33.795 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:18:34.498 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:18:34.547 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:18:37.933 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:18:37.970 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:18:38.938 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:18:39.405 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:18:42.800 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:17.593 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:17.633 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:17.814 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:17.856 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T09:19:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:19:17.963 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:19:18.167 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:18.167 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T09:19:18.167 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:19:18.167 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:19:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:19:18.816 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:19:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:19:18.817 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:19:18.978 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:19:19.156 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T09:19:19.240 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:19.240 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.241 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.241 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T09:19:19.241 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T09:19:19.241 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T09:19:19.241 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T09:19:19.241 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T09:19:19.299 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:19.300 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.300 DEBUG:teuthology.orchestra.run.smithi096:> sudo rpm -qi grub2-tools 2024-09-11T09:19:19.304 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T09:19:19.336 INFO:teuthology.orchestra.run.smithi096.stdout:Name : grub2-tools 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Epoch : 1 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Version : 2.06 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Release : 82.el9 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: x86_64 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Group : Unspecified 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Size : 8274670 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:License : GPLv3+ 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T09:19:19.337 INFO:teuthology.orchestra.run.smithi096.stdout:Packager : builder@centos.org 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:Vendor : CentOS 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:Summary : Support tools for GRUB. 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:Description : 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:hardware devices. 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:19:19.338 INFO:teuthology.orchestra.run.smithi096.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T09:19:19.339 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T09:19:19.339 INFO:teuthology.task.kernel:Updating grub on smithi096 to boot 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.339 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T09:19:19.390 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:19.390 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.390 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.390 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T09:19:19.390 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T09:19:19.390 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T09:19:19.390 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T09:19:19.390 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T09:19:19.449 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:19.450 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo rpm -qi grub2-tools 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Name : grub2-tools 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Epoch : 1 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Version : 2.06 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Release : 82.el9 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: x86_64 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Group : Unspecified 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Size : 8274670 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:License : GPLv3+ 2024-09-11T09:19:19.489 INFO:teuthology.orchestra.run.smithi017.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Packager : builder@centos.org 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Vendor : CentOS 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Summary : Support tools for GRUB. 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:Description : 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:hardware devices. 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:19:19.490 INFO:teuthology.orchestra.run.smithi017.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T09:19:19.491 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T09:19:19.491 INFO:teuthology.task.kernel:Updating grub on smithi017 to boot 5.14.0-505.el9.x86_64 2024-09-11T09:19:19.491 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T09:19:20.004 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-09-11T09:19:20.167 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-09-11T09:19:21.199 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:21.238 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:21.336 INFO:teuthology.orchestra.run.smithi096.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T09:19:21.379 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-09-11T09:19:21.381 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-11T09:19:21.396 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.AB13jMgLTC 2024-09-11T09:19:21.397 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AB13jMgLTC 2024-09-11T09:19:21.464 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.AB13jMgLTC 2024-09-11T09:19:21.468 INFO:teuthology.orchestra.run.smithi017.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T09:19:21.499 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-09-11T09:19:21.500 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-11T09:19:21.515 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.7pkidQFhD1 2024-09-11T09:19:21.516 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7pkidQFhD1 2024-09-11T09:19:21.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.7pkidQFhD1 2024-09-11T09:19:21.597 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:21.597 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T09:19:21.597 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T09:19:21.597 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T09:19:21.598 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.AB13jMgLTC is 6KB 2024-09-11T09:19:21.647 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.AB13jMgLTC 2024-09-11T09:19:21.662 DEBUG:teuthology.orchestra.run.smithi096:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T09:19:21.724 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T09:19:21.724 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T09:19:21.724 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T09:19:21.725 INFO:teuthology.orchestra.run.smithi096.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T09:19:21.726 DEBUG:teuthology.orchestra.run.smithi096:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T09:19:21.736 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.7pkidQFhD1 is 6KB 2024-09-11T09:19:21.756 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.7pkidQFhD1 2024-09-11T09:19:21.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T09:19:21.837 INFO:teuthology.orchestra.run.smithi017.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T09:19:21.837 INFO:teuthology.orchestra.run.smithi017.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T09:19:21.837 INFO:teuthology.orchestra.run.smithi017.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T09:19:21.837 INFO:teuthology.orchestra.run.smithi017.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T09:19:21.839 DEBUG:teuthology.orchestra.run.smithi017:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T09:19:21.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-09-11T09:19:21.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T09:19:22.449 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T09:19:22.450 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:19:22.450 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:19:22.818 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T09:19:22.902 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:22.903 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T09:19:22.903 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T09:19:22.903 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T09:19:22.903 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T09:19:22.903 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T09:19:22.903 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T09:19:22.903 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T09:19:22.962 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T09:19:22.962 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T09:19:22.962 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 82.el9 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8274670 2024-09-11T09:19:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T09:19:23.005 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T09:19:23.006 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T09:19:23.006 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-09-11T09:19:23.006 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:19:23.006 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T09:19:23.007 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T09:19:23.007 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 5.14.0-505.el9.x86_64 2024-09-11T09:19:23.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T09:19:23.653 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-11T09:19:24.970 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T09:19:25.007 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-11T09:19:25.009 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-11T09:19:25.025 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.E9uEqhe5JF 2024-09-11T09:19:25.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.E9uEqhe5JF 2024-09-11T09:19:25.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.E9uEqhe5JF 2024-09-11T09:19:25.239 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.E9uEqhe5JF is 6KB 2024-09-11T09:19:25.287 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.E9uEqhe5JF 2024-09-11T09:19:25.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T09:19:25.365 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T09:19:25.365 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T09:19:25.365 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T09:19:25.365 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T09:19:25.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T09:19:25.535 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-11T09:19:51.882 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T09:19:51.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:19:51.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:19:51.988 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T09:19:51.988 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-11T09:19:51.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:19:55.541 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T09:19:55.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:19:55.542 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:20:10.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-11T09:20:10.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-09-11T09:20:13.885 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-11T09:20:19.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:20:19.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:20:19.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-11T09:20:19.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:20:22.369 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-11T09:20:22.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:20:22.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:20:25.949 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-11T09:20:34.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:20:34.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:20:37.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:20:37.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:20:52.801 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-11T09:20:56.385 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-11T09:21:07.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:21:07.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:21:08.382 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-11T09:21:08.925 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:21:08.925 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T09:21:08.925 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-11T09:21:08.966 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-505.el9.x86_64 2024-09-11T09:21:08.966 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T09:21:08.966 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T09:21:08.966 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T09:21:09.967 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T09:21:09.967 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T09:21:10.024 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-09-11T09:21:10.174 DEBUG:teuthology.parallel:result is None 2024-09-11T09:21:11.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:21:11.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:21:11.741 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-11T09:21:12.085 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:21:12.085 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T09:21:12.085 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-11T09:21:12.100 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-505.el9.x86_64 2024-09-11T09:21:12.100 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T09:21:12.100 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T09:21:12.100 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T09:21:13.101 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-11T09:21:13.102 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T09:21:13.204 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-09-11T09:21:13.229 DEBUG:teuthology.parallel:result is None 2024-09-11T09:21:19.561 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T09:21:31.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-11T09:21:31.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:21:31.907 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-09-11T09:21:32.241 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-11T09:21:32.241 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T09:21:32.241 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-09-11T09:21:32.258 INFO:teuthology.orchestra.run.smithi017.stdout:5.14.0-505.el9.x86_64 2024-09-11T09:21:32.258 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T09:21:32.258 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T09:21:32.258 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T09:21:33.259 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T09:21:33.259 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T09:21:33.344 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2024-09-11T09:21:33.363 DEBUG:teuthology.parallel:result is None 2024-09-11T09:21:33.363 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T09:21:33.376 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T09:21:33.377 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T09:21:33.380 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T09:21:33.383 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T09:21:33.403 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T09:21:33.412 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T09:21:33.420 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T09:21:33.420 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T09:21:33.440 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T09:21:33.442 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T09:21:33.475 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T09:21:33.481 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T09:21:33.482 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T09:21:33.506 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T09:21:33.509 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T09:21:33.539 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T09:21:33.542 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T09:21:33.547 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T09:21:33.551 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T09:21:33.554 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T09:21:33.559 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T09:21:33.561 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T09:21:33.568 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T09:21:33.568 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T09:21:33.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T09:21:33.602 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T09:21:33.630 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T09:21:33.640 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T09:21:33.640 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T09:21:33.670 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T09:21:33.672 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T09:21:33.689 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T09:21:33.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T09:21:33.821 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T09:21:33.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T09:21:33.949 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:21:33.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T09:21:34.017 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T09:21:34.062 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T09:21:34.110 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T09:21:34.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T09:21:34.239 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:21:34.239 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T09:21:34.307 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T09:21:34.353 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T09:21:34.412 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T09:21:34.456 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T09:21:34.496 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:21:34.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T09:21:34.562 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-09-11T09:21:34.565 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-11T09:21:34.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-11T09:21:34.608 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:21:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:21:34.647 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:21:35.121 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T09:21:35.129 INFO:teuthology.task.internal:Starting timer... 2024-09-11T09:21:35.129 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T09:21:35.138 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T09:21:35.145 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-11T09:21:35.145 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2024-09-11T09:21:35.187 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2024-09-11T09:21:35.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-09-11T09:21:35.253 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-09-11T09:21:35.255 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-11T09:21:35.337 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-11T09:21:35.339 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T09:21:35.339 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2024-09-11T09:21:35.362 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2024-09-11T09:21:35.363 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-09-11T09:21:35.394 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-09-11T09:21:35.394 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-11T09:21:35.420 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-11T09:21:35.420 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2024-09-11T09:21:35.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T09:21:35.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:21:35.449 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T09:21:35.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:21:35.478 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T09:21:35.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:21:35.505 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T09:21:35.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2024-09-11T09:21:35.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-09-11T09:21:35.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-11T09:21:35.615 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T09:21:35.625 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T09:21:35.628 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T09:21:35.744 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T09:21:35.755 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T09:21:35.756 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T09:25:21.903 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2024-09-11T09:25:21.904 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-11T09:25:21.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:25:21.983 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-09-11T09:25:22.080 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-11T09:25:22.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:25:22.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:25:22.153 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-11T09:25:22.236 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-11T09:25:22.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:25:22.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T09:25:22.311 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-11T09:25:22.396 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-11T09:25:22.397 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T09:25:22.406 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T09:25:22.407 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T09:25:22.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:25:22.410 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T09:25:22.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:25:22.412 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T09:25:22.412 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:25:22.439 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T09:25:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T09:25:22.454 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T09:25:22.457 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T09:25:22.466 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T09:25:22.478 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-09-11T09:25:22.480 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T09:25:22.484 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2024-09-11T09:25:22.489 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-09-11T09:25:22.496 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2024-09-11T09:25:22.503 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T09:25:22.505 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-11T09:25:22.511 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T09:25:22.516 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-11T09:25:22.517 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T09:25:22.526 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T09:25:22.530 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T09:25:22.544 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T09:25:22.573 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:25:22.578 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:25:22.578 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-11T09:25:22.578 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.578 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.578 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.578 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.582 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:25:22.586 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:25:22.586 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-09-11T09:25:22.586 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.587 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.587 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.587 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.599 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:25:22.602 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:25:22.602 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-11T09:25:22.602 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.602 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.602 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.602 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:25:22.603 INFO:teuthology.run_tasks:Running task install... 2024-09-11T09:25:22.611 DEBUG:teuthology.task.install:project ceph 2024-09-11T09:25:22.612 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T09:25:22.612 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T09:25:22.612 INFO:teuthology.task.install:Using flavor: default 2024-09-11T09:25:22.617 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T09:25:22.617 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-11T09:25:22.618 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T09:25:22.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:25:22.620 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T09:25:22.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:25:22.621 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T09:25:22.622 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:25:22.760 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T09:25:22.760 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T09:25:22.772 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T09:25:22.773 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T09:25:22.781 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T09:25:22.781 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T09:25:22.874 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T09:25:22.874 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:25:22.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T09:25:22.882 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T09:25:22.882 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:25:22.882 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T09:25:22.891 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T09:25:22.891 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:25:22.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T09:25:22.902 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-11T09:25:22.902 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T09:25:22.907 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-11T09:25:22.907 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T09:25:22.918 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-11T09:25:22.918 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T09:25:22.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T09:25:22.973 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T09:25:22.984 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T09:25:23.045 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T09:25:23.048 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T09:25:23.059 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T09:25:23.111 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2024-09-11T09:25:23.112 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2024-09-11T09:25:23.113 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-11T09:25:23.114 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-11T09:25:23.123 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-09-11T09:25:23.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-09-11T09:25:23.339 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-11T09:25:23.351 INFO:teuthology.orchestra.run.smithi096.stdout:45 files removed 2024-09-11T09:25:23.364 INFO:teuthology.orchestra.run.smithi017.stdout:45 files removed 2024-09-11T09:25:23.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-09-11T09:25:23.377 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw 2024-09-11T09:25:23.391 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw 2024-09-11T09:25:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-11T09:25:23.979 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 288 kB/s | 84 kB 00:00 2024-09-11T09:25:23.993 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 249 kB/s | 84 kB 00:00 2024-09-11T09:25:24.222 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-11T09:25:24.271 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-11T09:25:24.285 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-11T09:25:24.505 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-11T09:25:24.541 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T09:25:24.552 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T09:25:25.286 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-11T09:25:26.782 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-11T09:25:26.954 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-11T09:25:28.854 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-09-11T09:25:31.063 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-11T09:25:31.196 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.5 MB 00:00 2024-09-11T09:25:32.617 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-11T09:25:32.964 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-11T09:25:33.844 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-11T09:25:35.681 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.5 MB 00:00 2024-09-11T09:25:37.173 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 65 kB/s | 19 kB 00:00 2024-09-11T09:25:37.919 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-11T09:25:39.209 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-11T09:25:39.299 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T09:25:40.685 INFO:teuthology.orchestra.run.smithi017.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-11T09:25:41.439 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-11T09:25:41.720 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:25:41.751 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:25:41.751 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T09:25:41.752 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T09:25:41.753 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:25:41.754 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:25:41.755 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:25:41.755 INFO:teuthology.orchestra.run.smithi096.stdout:Install 30 Packages 2024-09-11T09:25:41.755 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 2 Packages 2024-09-11T09:25:41.755 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:25:41.755 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 60 M 2024-09-11T09:25:41.755 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:25:42.633 INFO:teuthology.orchestra.run.smithi096.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.4 MB 00:00 2024-09-11T09:25:42.684 INFO:teuthology.orchestra.run.smithi096.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 505 kB/s | 25 kB 00:00 2024-09-11T09:25:42.784 INFO:teuthology.orchestra.run.smithi096.stdout:(3/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 7.4 MB/s | 752 kB 00:00 2024-09-11T09:25:42.876 INFO:teuthology.orchestra.run.smithi096.stdout:(4/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 5.4 MB/s | 506 kB 00:00 2024-09-11T09:25:43.210 INFO:teuthology.orchestra.run.smithi096.stdout:(5/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 10 MB/s | 11 MB 00:01 2024-09-11T09:25:43.293 INFO:teuthology.orchestra.run.smithi096.stdout:(6/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_64 13 MB/s | 5.3 MB 00:00 2024-09-11T09:25:43.318 INFO:teuthology.orchestra.run.smithi096.stdout:(7/32): python3-ceph-argparse-19.1.1-298.gb53b6 416 kB/s | 45 kB 00:00 2024-09-11T09:25:43.341 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T09:25:43.344 INFO:teuthology.orchestra.run.smithi096.stdout:(8/32): python3-ceph-common-19.1.1-298.gb53b6d3 2.6 MB/s | 136 kB 00:00 2024-09-11T09:25:43.369 INFO:teuthology.orchestra.run.smithi096.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 3.2 MB/s | 162 kB 00:00 2024-09-11T09:25:43.402 INFO:teuthology.orchestra.run.smithi096.stdout:(10/32): python3-rados-19.1.1-298.gb53b6d3e.el9 5.4 MB/s | 321 kB 00:00 2024-09-11T09:25:43.428 INFO:teuthology.orchestra.run.smithi096.stdout:(11/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-11T09:25:43.453 INFO:teuthology.orchestra.run.smithi096.stdout:(12/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 1.9 MB/s | 100 kB 00:00 2024-09-11T09:25:43.662 INFO:teuthology.orchestra.run.smithi096.stdout:(13/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 21 MB 00:01 2024-09-11T09:25:43.712 INFO:teuthology.orchestra.run.smithi096.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 377 kB/s | 107 kB 00:00 2024-09-11T09:25:43.780 INFO:teuthology.orchestra.run.smithi096.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-11T09:25:43.805 INFO:teuthology.orchestra.run.smithi096.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 456 kB/s | 160 kB 00:00 2024-09-11T09:25:43.855 INFO:teuthology.orchestra.run.smithi096.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 848 kB/s | 42 kB 00:00 2024-09-11T09:25:43.905 INFO:teuthology.orchestra.run.smithi096.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-11T09:25:43.930 INFO:teuthology.orchestra.run.smithi096.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-11T09:25:43.956 INFO:teuthology.orchestra.run.smithi096.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 715 kB/s | 36 kB 00:00 2024-09-11T09:25:43.981 INFO:teuthology.orchestra.run.smithi096.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-11T09:25:44.031 INFO:teuthology.orchestra.run.smithi096.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 4.0 MB/s | 309 kB 00:00 2024-09-11T09:25:44.090 INFO:teuthology.orchestra.run.smithi096.stdout:(23/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-09-11T09:25:44.115 INFO:teuthology.orchestra.run.smithi096.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 317 kB/s | 26 kB 00:00 2024-09-11T09:25:44.149 INFO:teuthology.orchestra.run.smithi096.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-11T09:25:44.183 INFO:teuthology.orchestra.run.smithi096.stdout:(26/32): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-09-11T09:25:44.216 INFO:teuthology.orchestra.run.smithi096.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-11T09:25:44.250 INFO:teuthology.orchestra.run.smithi096.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.3 MB/s | 839 kB 00:00 2024-09-11T09:25:44.350 INFO:teuthology.orchestra.run.smithi096.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-11T09:25:44.392 INFO:teuthology.orchestra.run.smithi096.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-09-11T09:25:44.467 INFO:teuthology.orchestra.run.smithi096.stdout:(31/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 3.4 MB 00:00 2024-09-11T09:25:44.559 INFO:teuthology.orchestra.run.smithi096.stdout:(32/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 15 MB/s | 3.2 MB 00:00 2024-09-11T09:25:44.565 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:25:44.565 INFO:teuthology.orchestra.run.smithi096.stdout:Total 22 MB/s | 60 MB 00:02 2024-09-11T09:25:44.673 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:25:44.709 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:25:44.709 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:25:45.409 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:25:45.409 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:25:45.767 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:25:45.782 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:25:45.782 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:25:45.782 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:25:45.782 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T09:25:45.783 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T09:25:45.784 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T09:25:45.785 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout:Install 30 Packages 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 60 M 2024-09-11T09:25:45.786 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:25:46.342 INFO:teuthology.orchestra.run.smithi107.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 5.4 MB 00:00 2024-09-11T09:25:46.409 INFO:teuthology.orchestra.run.smithi107.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 378 kB/s | 25 kB 00:00 2024-09-11T09:25:46.501 INFO:teuthology.orchestra.run.smithi107.stdout:(3/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 8.0 MB/s | 752 kB 00:00 2024-09-11T09:25:46.609 INFO:teuthology.orchestra.run.smithi107.stdout:(4/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 4.6 MB/s | 506 kB 00:00 2024-09-11T09:25:46.726 INFO:teuthology.orchestra.run.smithi107.stdout:(5/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 11 MB 00:00 2024-09-11T09:25:46.768 INFO:teuthology.orchestra.run.smithi107.stdout:(6/32): python3-ceph-argparse-19.1.1-298.gb53b6 1.1 MB/s | 45 kB 00:00 2024-09-11T09:25:46.877 INFO:teuthology.orchestra.run.smithi107.stdout:(7/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_64 20 MB/s | 5.3 MB 00:00 2024-09-11T09:25:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:25:46.902 INFO:teuthology.orchestra.run.smithi107.stdout:(8/32): python3-ceph-common-19.1.1-298.gb53b6d3 1.0 MB/s | 136 kB 00:00 2024-09-11T09:25:46.927 INFO:teuthology.orchestra.run.smithi107.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 3.2 MB/s | 162 kB 00:00 2024-09-11T09:25:46.961 INFO:teuthology.orchestra.run.smithi107.stdout:(10/32): python3-rados-19.1.1-298.gb53b6d3e.el9 5.4 MB/s | 321 kB 00:00 2024-09-11T09:25:46.982 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-11T09:25:46.986 INFO:teuthology.orchestra.run.smithi107.stdout:(11/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-11T09:25:47.011 INFO:teuthology.orchestra.run.smithi107.stdout:(12/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 1.9 MB/s | 100 kB 00:00 2024-09-11T09:25:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-11T09:25:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-11T09:25:47.053 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T09:25:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T09:25:47.105 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T09:25:47.220 INFO:teuthology.orchestra.run.smithi107.stdout:(13/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-11T09:25:47.282 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T09:25:47.329 INFO:teuthology.orchestra.run.smithi107.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 419 kB/s | 45 kB 00:00 2024-09-11T09:25:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T09:25:47.354 INFO:teuthology.orchestra.run.smithi107.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 291 kB/s | 107 kB 00:00 2024-09-11T09:25:47.379 INFO:teuthology.orchestra.run.smithi107.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 436 kB/s | 160 kB 00:00 2024-09-11T09:25:47.429 INFO:teuthology.orchestra.run.smithi107.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 847 kB/s | 42 kB 00:00 2024-09-11T09:25:47.455 INFO:teuthology.orchestra.run.smithi107.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-11T09:25:47.497 INFO:teuthology.orchestra.run.smithi107.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-11T09:25:47.522 INFO:teuthology.orchestra.run.smithi107.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 536 kB/s | 36 kB 00:00 2024-09-11T09:25:47.547 INFO:teuthology.orchestra.run.smithi107.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-11T09:25:47.573 INFO:teuthology.orchestra.run.smithi107.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-09-11T09:25:47.640 INFO:teuthology.orchestra.run.smithi107.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.6 MB/s | 309 kB 00:00 2024-09-11T09:25:47.665 INFO:teuthology.orchestra.run.smithi107.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-11T09:25:47.707 INFO:teuthology.orchestra.run.smithi107.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 218 kB/s | 26 kB 00:00 2024-09-11T09:25:47.727 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-11T09:25:47.732 INFO:teuthology.orchestra.run.smithi107.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-11T09:25:47.757 INFO:teuthology.orchestra.run.smithi107.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-11T09:25:47.791 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:25:47.808 INFO:teuthology.orchestra.run.smithi107.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-11T09:25:47.866 INFO:teuthology.orchestra.run.smithi107.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T09:25:47.884 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:25:47.941 INFO:teuthology.orchestra.run.smithi107.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-11T09:25:47.972 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-11T09:25:48.002 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T09:25:48.050 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T09:25:48.075 INFO:teuthology.orchestra.run.smithi107.stdout:(31/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 3.4 MB 00:00 2024-09-11T09:25:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-11T09:25:48.125 INFO:teuthology.orchestra.run.smithi107.stdout:(32/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-11T09:25:48.131 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:25:48.131 INFO:teuthology.orchestra.run.smithi107.stdout:Total 26 MB/s | 60 MB 00:02 2024-09-11T09:25:48.145 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-11T09:25:48.214 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T09:25:48.233 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:25:48.237 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:25:48.269 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:25:48.269 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:25:48.302 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:25:48.336 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-11T09:25:48.442 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-11T09:25:48.471 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-11T09:25:48.512 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-11T09:25:48.552 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-11T09:25:48.581 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-11T09:25:48.614 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-11T09:25:48.941 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-11T09:25:48.946 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:25:48.946 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:25:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T09:25:49.076 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T09:25:49.110 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-11T09:25:49.152 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-11T09:25:49.567 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:25:49.577 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:25:49.577 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:25:49.578 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:25:49.578 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:25:49.578 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T09:25:49.578 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2024-09-11T09:25:49.578 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T09:25:49.579 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T09:25:49.579 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:25:49.579 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T09:25:49.579 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T09:25:49.579 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T09:25:49.580 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T09:25:49.580 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T09:25:49.580 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T09:25:49.580 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T09:25:49.580 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T09:25:49.581 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout:Installing weak dependencies: 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:25:49.582 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:25:49.583 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:25:49.583 INFO:teuthology.orchestra.run.smithi017.stdout:Install 30 Packages 2024-09-11T09:25:49.583 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 2 Packages 2024-09-11T09:25:49.583 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:25:49.583 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 60 M 2024-09-11T09:25:49.583 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:25:49.651 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-11T09:25:49.680 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T09:25:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T09:25:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-11T09:25:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-11T09:25:50.401 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:25:50.484 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-11T09:25:50.497 INFO:teuthology.orchestra.run.smithi017.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 9.4 MB/s | 5.4 MB 00:00 2024-09-11T09:25:50.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-11T09:25:50.555 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-11T09:25:50.578 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T09:25:50.580 INFO:teuthology.orchestra.run.smithi017.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 303 kB/s | 25 kB 00:00 2024-09-11T09:25:50.603 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T09:25:50.739 INFO:teuthology.orchestra.run.smithi017.stdout:(3/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 4.6 MB/s | 752 kB 00:00 2024-09-11T09:25:50.796 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T09:25:50.866 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T09:25:50.897 INFO:teuthology.orchestra.run.smithi017.stdout:(4/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 11 MB 00:00 2024-09-11T09:25:50.931 INFO:teuthology.orchestra.run.smithi017.stdout:(5/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 2.6 MB/s | 506 kB 00:00 2024-09-11T09:25:51.139 INFO:teuthology.orchestra.run.smithi017.stdout:(6/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 21 MB 00:01 2024-09-11T09:25:51.190 INFO:teuthology.orchestra.run.smithi017.stdout:(7/32): python3-ceph-argparse-19.1.1-298.gb53b6 174 kB/s | 45 kB 00:00 2024-09-11T09:25:51.215 INFO:teuthology.orchestra.run.smithi017.stdout:(8/32): python3-ceph-common-19.1.1-298.gb53b6d3 1.8 MB/s | 136 kB 00:00 2024-09-11T09:25:51.248 INFO:teuthology.orchestra.run.smithi017.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 2.7 MB/s | 162 kB 00:00 2024-09-11T09:25:51.274 INFO:teuthology.orchestra.run.smithi017.stdout:(10/32): python3-rados-19.1.1-298.gb53b6d3e.el9 5.4 MB/s | 321 kB 00:00 2024-09-11T09:25:51.274 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-11T09:25:51.299 INFO:teuthology.orchestra.run.smithi017.stdout:(11/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-11T09:25:51.323 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:25:51.324 INFO:teuthology.orchestra.run.smithi017.stdout:(12/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 1.9 MB/s | 100 kB 00:00 2024-09-11T09:25:51.411 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:25:51.417 INFO:teuthology.orchestra.run.smithi017.stdout:(13/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_6 10 MB/s | 5.3 MB 00:00 2024-09-11T09:25:51.458 INFO:teuthology.orchestra.run.smithi017.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 673 kB/s | 107 kB 00:00 2024-09-11T09:25:51.484 INFO:teuthology.orchestra.run.smithi017.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-11T09:25:51.505 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-11T09:25:51.509 INFO:teuthology.orchestra.run.smithi017.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-09-11T09:25:51.542 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T09:25:51.567 INFO:teuthology.orchestra.run.smithi017.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 725 kB/s | 42 kB 00:00 2024-09-11T09:25:51.593 INFO:teuthology.orchestra.run.smithi017.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-11T09:25:51.597 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T09:25:51.618 INFO:teuthology.orchestra.run.smithi017.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-11T09:25:51.642 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-11T09:25:51.660 INFO:teuthology.orchestra.run.smithi017.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 538 kB/s | 36 kB 00:00 2024-09-11T09:25:51.685 INFO:teuthology.orchestra.run.smithi017.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-11T09:25:51.687 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-11T09:25:51.710 INFO:teuthology.orchestra.run.smithi017.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-09-11T09:25:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T09:25:51.769 INFO:teuthology.orchestra.run.smithi017.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-11T09:25:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:25:51.794 INFO:teuthology.orchestra.run.smithi017.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-11T09:25:51.820 INFO:teuthology.orchestra.run.smithi017.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 279 kB/s | 26 kB 00:00 2024-09-11T09:25:51.845 INFO:teuthology.orchestra.run.smithi017.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-11T09:25:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:25:51.879 INFO:teuthology.orchestra.run.smithi017.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-11T09:25:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-11T09:25:51.929 INFO:teuthology.orchestra.run.smithi017.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-11T09:25:51.979 INFO:teuthology.orchestra.run.smithi017.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T09:25:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-11T09:25:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-11T09:25:52.063 INFO:teuthology.orchestra.run.smithi017.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-11T09:25:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-11T09:25:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-11T09:25:52.143 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:25:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-11T09:25:52.172 INFO:teuthology.orchestra.run.smithi017.stdout:(31/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 16 MB/s | 3.2 MB 00:00 2024-09-11T09:25:52.179 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:25:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-11T09:25:52.305 INFO:teuthology.orchestra.run.smithi017.stdout:(32/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 9.1 MB/s | 3.4 MB 00:00 2024-09-11T09:25:52.311 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:25:52.311 INFO:teuthology.orchestra.run.smithi017.stdout:Total 22 MB/s | 60 MB 00:02 2024-09-11T09:25:52.425 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:25:52.467 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:25:52.468 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:25:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-11T09:25:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T09:25:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T09:25:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-11T09:25:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-11T09:25:52.747 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:25:52.791 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T09:25:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T09:25:52.864 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T09:25:52.864 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T09:25:52.864 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:25:52.901 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:25:53.166 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:25:53.166 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:25:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-11T09:25:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T09:25:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T09:25:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-11T09:25:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-11T09:25:54.738 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:25:54.875 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-11T09:25:54.902 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-11T09:25:54.927 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-11T09:25:54.949 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T09:25:54.977 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-11T09:25:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T09:25:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-11T09:25:55.572 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-11T09:25:55.679 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:25:55.755 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:25:55.756 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:25:55.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:25:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-11T09:25:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T09:25:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-11T09:25:55.989 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-11T09:25:56.033 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-11T09:25:56.104 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T09:25:56.133 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:25:56.197 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:25:56.231 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-11T09:25:56.343 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-11T09:25:56.368 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:25:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-11T09:25:56.412 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T09:25:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-11T09:25:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-11T09:25:56.493 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-11T09:25:56.507 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T09:25:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T09:25:56.507 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T09:25:56.507 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:25:56.526 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-11T09:25:56.540 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:25:56.877 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-11T09:25:56.922 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T09:25:57.014 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-11T09:25:57.051 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-11T09:25:57.097 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-11T09:25:57.604 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-11T09:25:57.633 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T09:25:57.712 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-11T09:25:57.793 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-11T09:25:57.818 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-11T09:26:00.212 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:26:00.260 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:26:00.849 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-11T09:26:00.885 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T09:26:00.979 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-11T09:26:00.979 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T09:26:00.979 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T09:26:00.979 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:01.011 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:03.216 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:03.216 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-09-11T09:26:03.216 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-09-11T09:26:03.217 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-09-11T09:26:03.217 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-09-11T09:26:03.217 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-09-11T09:26:03.217 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-09-11T09:26:03.217 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-09-11T09:26:03.217 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:04.198 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T09:26:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T09:26:04.224 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:04.224 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T09:26:04.224 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T09:26:04.224 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T09:26:04.224 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:04.226 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T09:26:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T09:26:04.317 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:05.451 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-11T09:26:05.452 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-11T09:26:05.453 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-11T09:26:05.454 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-11T09:26:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:06.466 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:06.466 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.467 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.468 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:06.469 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:06.561 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:06.561 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-11T09:26:06.561 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-11T09:26:06.561 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-11T09:26:06.562 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-11T09:26:06.562 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-11T09:26:06.562 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-11T09:26:06.562 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-11T09:26:06.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:06.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-test 2024-09-11T09:26:07.427 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:07.546 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T09:26:07.558 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T09:26:07.559 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout:Install 4 Packages 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 49 M 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 209 M 2024-09-11T09:26:07.560 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T09:26:07.574 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:07.574 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T09:26:07.574 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T09:26:07.574 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T09:26:07.574 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:07.576 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T09:26:07.677 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T09:26:07.677 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:08.040 INFO:teuthology.orchestra.run.smithi096.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-11T09:26:08.090 INFO:teuthology.orchestra.run.smithi096.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T09:26:08.264 INFO:teuthology.orchestra.run.smithi096.stdout:(3/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 361 kB/s | 163 kB 00:00 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-11T09:26:08.847 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-11T09:26:08.848 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-11T09:26:08.849 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-11T09:26:09.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:09.778 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:09.778 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-11T09:26:09.778 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.778 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.778 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T09:26:09.779 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T09:26:09.780 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T09:26:09.781 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T09:26:09.781 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T09:26:09.781 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T09:26:09.781 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:09.781 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:10.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-09-11T09:26:10.710 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:27 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:10.851 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:10.851 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:10.851 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:10.852 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-11T09:26:10.853 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:10.853 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 49 M 2024-09-11T09:26:10.853 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 209 M 2024-09-11T09:26:10.853 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:11.330 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 739 kB/s | 163 kB 00:00 2024-09-11T09:26:11.373 INFO:teuthology.orchestra.run.smithi096.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 14 MB/s | 49 MB 00:03 2024-09-11T09:26:11.375 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:11.376 INFO:teuthology.orchestra.run.smithi096.stdout:Total 13 MB/s | 49 MB 00:03 2024-09-11T09:26:11.404 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:26:11.424 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:26:11.424 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:26:11.568 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /root 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2024-09-11T09:26:11.569 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:11.705 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-11T09:26:11.756 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 470 kB/s | 304 kB 00:00 2024-09-11T09:26:11.807 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:26:11.807 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:26:12.341 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:26:12.449 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T09:26:12.488 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T09:26:12.517 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T09:26:12.605 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T09:26:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-11T09:26:12.633 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:12.633 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T09:26:12.633 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T09:26:12.633 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T09:26:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T09:26:12.639 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-11T09:26:12.642 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:12.642 INFO:teuthology.orchestra.run.smithi107.stdout:Total 27 MB/s | 49 MB 00:01 2024-09-11T09:26:12.669 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:26:12.688 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:26:12.688 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:26:12.725 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-11T09:26:12.725 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:13.069 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:26:13.070 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:26:13.601 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:26:13.684 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T09:26:13.722 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T09:26:13.766 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T09:26:13.858 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-11T09:26:13.859 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-11T09:26:13.860 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-11T09:26:13.861 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-11T09:26:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-11T09:26:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:14.908 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2024-09-11T09:26:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.908 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.908 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.909 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.910 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T09:26:14.911 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T09:26:14.912 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:14.912 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:15.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-test 2024-09-11T09:26:15.816 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:15.953 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T09:26:15.954 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout:Install 4 Packages 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 49 M 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 209 M 2024-09-11T09:26:15.955 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:26:16.197 INFO:teuthology.orchestra.run.smithi017.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-11T09:26:16.222 INFO:teuthology.orchestra.run.smithi017.stdout:(2/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 751 kB/s | 163 kB 00:00 2024-09-11T09:26:16.248 INFO:teuthology.orchestra.run.smithi017.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T09:26:17.159 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T09:26:17.232 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T09:26:17.515 INFO:teuthology.orchestra.run.smithi017.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-11T09:26:17.517 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:17.517 INFO:teuthology.orchestra.run.smithi017.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-11T09:26:17.545 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:26:17.568 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:26:17.568 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:26:17.979 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:26:17.979 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:26:18.349 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T09:26:18.430 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T09:26:18.560 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:26:18.643 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T09:26:18.681 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T09:26:18.708 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T09:26:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T09:26:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-11T09:26:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-11T09:26:19.701 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T09:26:20.024 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:20.025 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:20.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph 2024-09-11T09:26:20.860 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T09:26:20.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-11T09:26:20.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-11T09:26:20.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T09:26:20.879 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:41 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:21.018 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:21.020 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:21.020 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:26:21.020 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:21.020 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T09:26:21.021 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T09:26:21.022 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T09:26:21.023 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout:Install 37 Packages 2024-09-11T09:26:21.024 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:21.025 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 30 M 2024-09-11T09:26:21.025 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 108 M 2024-09-11T09:26:21.025 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T09:26:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:21.249 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:21.249 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T09:26:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T09:26:21.250 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:21.250 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:21.401 INFO:teuthology.orchestra.run.smithi096.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-11T09:26:21.556 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-09-11T09:26:21.584 INFO:teuthology.orchestra.run.smithi096.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-11T09:26:21.643 INFO:teuthology.orchestra.run.smithi096.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 6.5 MB/s | 2.4 MB 00:00 2024-09-11T09:26:21.776 INFO:teuthology.orchestra.run.smithi096.stdout:(4/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-11T09:26:21.805 INFO:teuthology.orchestra.run.smithi096.stdout:(5/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 1.5 MB/s | 249 kB 00:00 2024-09-11T09:26:21.994 INFO:teuthology.orchestra.run.smithi096.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-11T09:26:22.142 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:39 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:22.186 INFO:teuthology.orchestra.run.smithi096.stdout:(7/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-11T09:26:22.211 INFO:teuthology.orchestra.run.smithi096.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-11T09:26:22.269 INFO:teuthology.orchestra.run.smithi096.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-09-11T09:26:22.287 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T09:26:22.290 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T09:26:22.291 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T09:26:22.292 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:22.293 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:22.294 INFO:teuthology.orchestra.run.smithi107.stdout:Install 37 Packages 2024-09-11T09:26:22.294 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:22.294 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 30 M 2024-09-11T09:26:22.294 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 108 M 2024-09-11T09:26:22.294 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:22.311 INFO:teuthology.orchestra.run.smithi096.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.2 MB 00:00 2024-09-11T09:26:22.337 INFO:teuthology.orchestra.run.smithi096.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-09-11T09:26:22.362 INFO:teuthology.orchestra.run.smithi096.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 768 kB/s | 135 kB 00:00 2024-09-11T09:26:22.446 INFO:teuthology.orchestra.run.smithi096.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 417 kB/s | 35 kB 00:00 2024-09-11T09:26:22.471 INFO:teuthology.orchestra.run.smithi096.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T09:26:22.505 INFO:teuthology.orchestra.run.smithi096.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 978 kB/s | 188 kB 00:00 2024-09-11T09:26:22.533 INFO:teuthology.orchestra.run.smithi096.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 917 kB/s | 77 kB 00:00 2024-09-11T09:26:22.555 INFO:teuthology.orchestra.run.smithi096.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-11T09:26:22.581 INFO:teuthology.orchestra.run.smithi096.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 550 kB/s | 42 kB 00:00 2024-09-11T09:26:22.623 INFO:teuthology.orchestra.run.smithi096.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-11T09:26:22.648 INFO:teuthology.orchestra.run.smithi096.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-11T09:26:22.681 INFO:teuthology.orchestra.run.smithi096.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-11T09:26:22.691 INFO:teuthology.orchestra.run.smithi107.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 52 kB/s | 6.5 kB 00:00 2024-09-11T09:26:22.706 INFO:teuthology.orchestra.run.smithi096.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-11T09:26:22.732 INFO:teuthology.orchestra.run.smithi096.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 220 kB/s | 11 kB 00:00 2024-09-11T09:26:22.757 INFO:teuthology.orchestra.run.smithi096.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-09-11T09:26:22.782 INFO:teuthology.orchestra.run.smithi096.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-11T09:26:22.808 INFO:teuthology.orchestra.run.smithi096.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-11T09:26:22.833 INFO:teuthology.orchestra.run.smithi096.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-11T09:26:22.858 INFO:teuthology.orchestra.run.smithi096.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-11T09:26:22.875 INFO:teuthology.orchestra.run.smithi107.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-11T09:26:22.892 INFO:teuthology.orchestra.run.smithi096.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-11T09:26:22.925 INFO:teuthology.orchestra.run.smithi107.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 6.6 MB/s | 2.4 MB 00:00 2024-09-11T09:26:22.926 INFO:teuthology.orchestra.run.smithi096.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-11T09:26:22.951 INFO:teuthology.orchestra.run.smithi096.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-09-11T09:26:22.976 INFO:teuthology.orchestra.run.smithi096.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-11T09:26:23.002 INFO:teuthology.orchestra.run.smithi096.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-11T09:26:23.017 INFO:teuthology.orchestra.run.smithi107.stdout:(4/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 2.7 MB/s | 249 kB 00:00 2024-09-11T09:26:23.028 INFO:teuthology.orchestra.run.smithi096.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-11T09:26:23.052 INFO:teuthology.orchestra.run.smithi096.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-11T09:26:23.078 INFO:teuthology.orchestra.run.smithi096.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T09:26:23.106 INFO:teuthology.orchestra.run.smithi096.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.0 MB/s | 427 kB 00:00 2024-09-11T09:26:23.109 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:23.110 INFO:teuthology.orchestra.run.smithi096.stdout:Total 15 MB/s | 30 MB 00:02 2024-09-11T09:26:23.118 INFO:teuthology.orchestra.run.smithi107.stdout:(5/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-11T09:26:23.193 INFO:teuthology.orchestra.run.smithi107.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-11T09:26:23.235 INFO:teuthology.orchestra.run.smithi107.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-11T09:26:23.253 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:26:23.292 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:26:23.293 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:26:23.419 INFO:teuthology.orchestra.run.smithi107.stdout:(8/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-11T09:26:23.444 INFO:teuthology.orchestra.run.smithi107.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 648 kB/s | 135 kB 00:00 2024-09-11T09:26:23.477 INFO:teuthology.orchestra.run.smithi107.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 3.4 MB/s | 1.2 MB 00:00 2024-09-11T09:26:23.503 INFO:teuthology.orchestra.run.smithi107.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-11T09:26:23.545 INFO:teuthology.orchestra.run.smithi107.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-11T09:26:23.579 INFO:teuthology.orchestra.run.smithi107.stdout:(13/37): python3-requests-2.25.1-8.el9.noarch.r 782 kB/s | 125 kB 00:00 2024-09-11T09:26:23.600 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-11T09:26:23.604 INFO:teuthology.orchestra.run.smithi107.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-11T09:26:23.629 INFO:teuthology.orchestra.run.smithi107.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 411 kB/s | 35 kB 00:00 2024-09-11T09:26:23.654 INFO:teuthology.orchestra.run.smithi107.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-11T09:26:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-11T09:26:23.685 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T09:26:23.705 INFO:teuthology.orchestra.run.smithi107.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-11T09:26:23.747 INFO:teuthology.orchestra.run.smithi107.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-11T09:26:23.772 INFO:teuthology.orchestra.run.smithi107.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-11T09:26:23.776 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:26:23.776 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:26:23.797 INFO:teuthology.orchestra.run.smithi107.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-11T09:26:23.823 INFO:teuthology.orchestra.run.smithi107.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 426 kB/s | 11 kB 00:00 2024-09-11T09:26:23.848 INFO:teuthology.orchestra.run.smithi107.stdout:(23/37): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-11T09:26:23.874 INFO:teuthology.orchestra.run.smithi107.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-11T09:26:23.907 INFO:teuthology.orchestra.run.smithi107.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 743 kB/s | 19 kB 00:00 2024-09-11T09:26:23.932 INFO:teuthology.orchestra.run.smithi107.stdout:(26/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-09-11T09:26:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 406 kB/s | 20 kB 00:00 2024-09-11T09:26:23.983 INFO:teuthology.orchestra.run.smithi107.stdout:(28/37): python3-jaraco-collections-3.0.0-8.el9 172 kB/s | 23 kB 00:00 2024-09-11T09:26:24.008 INFO:teuthology.orchestra.run.smithi107.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-11T09:26:24.033 INFO:teuthology.orchestra.run.smithi107.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-11T09:26:24.058 INFO:teuthology.orchestra.run.smithi107.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-11T09:26:24.084 INFO:teuthology.orchestra.run.smithi107.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-11T09:26:24.109 INFO:teuthology.orchestra.run.smithi107.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-11T09:26:24.134 INFO:teuthology.orchestra.run.smithi107.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-11T09:26:24.160 INFO:teuthology.orchestra.run.smithi107.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 793 kB/s | 20 kB 00:00 2024-09-11T09:26:24.185 INFO:teuthology.orchestra.run.smithi107.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T09:26:24.210 INFO:teuthology.orchestra.run.smithi107.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-11T09:26:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:24.214 INFO:teuthology.orchestra.run.smithi107.stdout:Total 16 MB/s | 30 MB 00:01 2024-09-11T09:26:24.365 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:26:24.405 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:26:24.405 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:26:24.898 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:26:24.899 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:26:25.179 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:26:25.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-11T09:26:25.293 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-11T09:26:25.329 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-11T09:26:25.402 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-11T09:26:25.453 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-11T09:26:25.489 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-11T09:26:25.579 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-11T09:26:25.665 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-11T09:26:25.714 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-11T09:26:25.749 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-11T09:26:25.787 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-11T09:26:25.822 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-11T09:26:25.855 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-11T09:26:25.898 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-11T09:26:25.948 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-11T09:26:25.982 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-11T09:26:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-11T09:26:26.084 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-11T09:26:26.084 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-11T09:26:26.084 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-11T09:26:26.085 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-11T09:26:26.085 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T09:26:26.145 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-11T09:26:26.270 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:26:26.292 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-11T09:26:26.339 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-11T09:26:26.372 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-11T09:26:26.380 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-11T09:26:26.399 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-11T09:26:26.423 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T09:26:26.423 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:26.423 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:26:26.423 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:26.423 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:26.424 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T09:26:26.424 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T09:26:26.424 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:26.424 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:26.433 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-11T09:26:26.471 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-11T09:26:26.472 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-11T09:26:26.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-11T09:26:26.529 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-11T09:26:26.543 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-11T09:26:26.607 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-11T09:26:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-11T09:26:26.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph 2024-09-11T09:26:26.731 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-11T09:26:26.779 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-11T09:26:26.796 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-11T09:26:26.823 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-11T09:26:26.839 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-11T09:26:26.861 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-11T09:26:26.896 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-11T09:26:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-11T09:26:26.929 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-11T09:26:26.970 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-11T09:26:27.007 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-11T09:26:27.013 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-11T09:26:27.055 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-11T09:26:27.056 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-11T09:26:27.105 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-11T09:26:27.147 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-11T09:26:27.183 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-11T09:26:27.227 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-11T09:26:27.281 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:40 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:27.344 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-11T09:26:27.374 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T09:26:27.375 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-11T09:26:27.400 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T09:26:27.400 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:27.400 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T09:26:27.400 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T09:26:27.400 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T09:26:27.400 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:27.428 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T09:26:27.431 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T09:26:27.432 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T09:26:27.433 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout:Install 37 Packages 2024-09-11T09:26:27.434 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:27.435 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 30 M 2024-09-11T09:26:27.435 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 108 M 2024-09-11T09:26:27.435 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:26:27.462 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-11T09:26:27.523 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-11T09:26:27.561 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-11T09:26:27.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-11T09:26:27.674 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-11T09:26:27.824 INFO:teuthology.orchestra.run.smithi017.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 49 kB/s | 6.5 kB 00:00 2024-09-11T09:26:27.863 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-11T09:26:27.916 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-11T09:26:28.003 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-11T09:26:28.041 INFO:teuthology.orchestra.run.smithi017.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 4.3 MB/s | 1.5 MB 00:00 2024-09-11T09:26:28.101 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-11T09:26:28.108 INFO:teuthology.orchestra.run.smithi017.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 5.7 MB/s | 2.4 MB 00:00 2024-09-11T09:26:28.140 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-11T09:26:28.167 INFO:teuthology.orchestra.run.smithi017.stdout:(4/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 4.2 MB/s | 249 kB 00:00 2024-09-11T09:26:28.271 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-11T09:26:28.375 INFO:teuthology.orchestra.run.smithi017.stdout:(5/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 8.5 MB/s | 4.7 MB 00:00 2024-09-11T09:26:28.441 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-11T09:26:28.471 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T09:26:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T09:26:28.498 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:28.498 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T09:26:28.498 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T09:26:28.498 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T09:26:28.498 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:28.626 INFO:teuthology.orchestra.run.smithi017.stdout:(6/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-11T09:26:28.801 INFO:teuthology.orchestra.run.smithi017.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 399 kB/s | 253 kB 00:00 2024-09-11T09:26:28.867 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T09:26:28.894 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T09:26:28.894 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:28.894 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T09:26:28.894 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T09:26:28.894 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T09:26:28.894 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:28.909 INFO:teuthology.orchestra.run.smithi017.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.2 MB/s | 135 kB 00:00 2024-09-11T09:26:29.026 INFO:teuthology.orchestra.run.smithi017.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.0 MB/s | 125 kB 00:00 2024-09-11T09:26:29.077 INFO:teuthology.orchestra.run.smithi017.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 236 kB/s | 106 kB 00:00 2024-09-11T09:26:29.143 INFO:teuthology.orchestra.run.smithi017.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-09-11T09:26:29.261 INFO:teuthology.orchestra.run.smithi017.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-11T09:26:29.286 INFO:teuthology.orchestra.run.smithi017.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 901 kB/s | 188 kB 00:00 2024-09-11T09:26:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T09:26:29.351 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T09:26:29.351 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:29.351 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T09:26:29.351 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T09:26:29.351 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T09:26:29.351 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:29.370 INFO:teuthology.orchestra.run.smithi017.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 319 kB/s | 35 kB 00:00 2024-09-11T09:26:29.395 INFO:teuthology.orchestra.run.smithi017.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 711 kB/s | 77 kB 00:00 2024-09-11T09:26:29.495 INFO:teuthology.orchestra.run.smithi017.stdout:(16/37): python3-cryptography-36.0.1-4.el9.x86_ 1.1 MB/s | 1.2 MB 00:01 2024-09-11T09:26:29.521 INFO:teuthology.orchestra.run.smithi017.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 339 kB/s | 51 kB 00:00 2024-09-11T09:26:29.546 INFO:teuthology.orchestra.run.smithi017.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 276 kB/s | 42 kB 00:00 2024-09-11T09:26:29.576 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T09:26:29.596 INFO:teuthology.orchestra.run.smithi017.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-11T09:26:29.600 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T09:26:29.600 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:29.600 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T09:26:29.601 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T09:26:29.601 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T09:26:29.601 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:29.644 INFO:teuthology.orchestra.run.smithi017.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 446 kB/s | 43 kB 00:00 2024-09-11T09:26:29.665 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T09:26:29.672 INFO:teuthology.orchestra.run.smithi017.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-09-11T09:26:29.697 INFO:teuthology.orchestra.run.smithi017.stdout:(22/37): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-09-11T09:26:29.722 INFO:teuthology.orchestra.run.smithi017.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-09-11T09:26:29.756 INFO:teuthology.orchestra.run.smithi017.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 931 kB/s | 23 kB 00:00 2024-09-11T09:26:29.781 INFO:teuthology.orchestra.run.smithi017.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 859 kB/s | 19 kB 00:00 2024-09-11T09:26:29.806 INFO:teuthology.orchestra.run.smithi017.stdout:(26/37): python3-jaraco-classes-3.2.1-5.el9.noa 162 kB/s | 18 kB 00:00 2024-09-11T09:26:29.831 INFO:teuthology.orchestra.run.smithi017.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-11T09:26:29.857 INFO:teuthology.orchestra.run.smithi017.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-11T09:26:29.882 INFO:teuthology.orchestra.run.smithi017.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-11T09:26:29.916 INFO:teuthology.orchestra.run.smithi017.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-11T09:26:29.939 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T09:26:29.941 INFO:teuthology.orchestra.run.smithi017.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 281 kB/s | 16 kB 00:00 2024-09-11T09:26:29.966 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T09:26:29.967 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:29.967 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T09:26:29.967 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T09:26:29.967 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T09:26:29.967 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:29.974 INFO:teuthology.orchestra.run.smithi017.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.0 MB/s | 36 kB 00:00 2024-09-11T09:26:30.000 INFO:teuthology.orchestra.run.smithi017.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-09-11T09:26:30.025 INFO:teuthology.orchestra.run.smithi017.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-11T09:26:30.050 INFO:teuthology.orchestra.run.smithi017.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 809 kB/s | 20 kB 00:00 2024-09-11T09:26:30.075 INFO:teuthology.orchestra.run.smithi017.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T09:26:30.101 INFO:teuthology.orchestra.run.smithi017.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-11T09:26:30.105 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:30.105 INFO:teuthology.orchestra.run.smithi017.stdout:Total 11 MB/s | 30 MB 00:02 2024-09-11T09:26:30.266 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:26:30.311 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:26:30.311 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:26:30.406 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T09:26:30.430 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T09:26:30.430 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:30.430 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T09:26:30.431 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T09:26:30.431 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T09:26:30.431 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:30.665 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T09:26:30.689 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T09:26:30.689 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:30.689 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T09:26:30.689 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T09:26:30.689 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T09:26:30.689 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:30.754 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T09:26:30.814 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:26:30.814 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:26:31.566 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T09:26:31.566 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-11T09:26:31.567 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-11T09:26:31.568 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-11T09:26:31.569 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-11T09:26:31.570 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-11T09:26:32.184 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:26:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-11T09:26:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-11T09:26:32.306 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-11T09:26:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-11T09:26:32.423 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-11T09:26:32.455 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:32.462 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T09:26:32.463 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T09:26:32.464 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:32.465 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:32.552 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-11T09:26:32.611 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-11T09:26:32.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-11T09:26:32.613 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-11T09:26:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-11T09:26:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-11T09:26:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-11T09:26:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-11T09:26:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-11T09:26:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-11T09:26:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-11T09:26:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-11T09:26:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-11T09:26:32.629 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-11T09:26:32.675 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-11T09:26:32.710 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-11T09:26:32.739 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-11T09:26:32.782 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-11T09:26:32.816 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-11T09:26:32.857 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-11T09:26:32.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-base 2024-09-11T09:26:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-11T09:26:32.929 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-11T09:26:32.978 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-11T09:26:33.025 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-11T09:26:33.102 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-11T09:26:33.264 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-11T09:26:33.354 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-11T09:26:33.414 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-11T09:26:33.451 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-11T09:26:33.461 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:33.533 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-11T09:26:33.550 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:26:33.592 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:33.593 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:26:33.593 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-11T09:26:33.643 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-11T09:26:33.643 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:33.643 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T09:26:33.644 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T09:26:33.645 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T09:26:33.646 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T09:26:33.647 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:33.647 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:33.657 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install cephadm 2024-09-11T09:26:33.802 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-11T09:26:33.847 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-11T09:26:33.928 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-11T09:26:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-11T09:26:34.054 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-09-11T09:26:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-11T09:26:34.186 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-11T09:26:34.217 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:34.348 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 765 k 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 771 k 2024-09-11T09:26:34.349 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:34.358 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-11T09:26:34.389 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T09:26:34.417 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-11T09:26:34.417 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:34.418 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T09:26:34.418 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T09:26:34.418 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T09:26:34.418 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:34.634 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 2.6 MB/s | 765 kB 00:00 2024-09-11T09:26:34.635 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:34.635 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:34.635 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.6 MB/s | 765 kB 00:00 2024-09-11T09:26:34.635 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:26:34.640 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:26:34.640 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:26:34.646 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:26:34.647 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:26:34.721 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:26:34.725 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:26:34.767 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:34.768 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:26:34.768 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:34.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-09-11T09:26:35.174 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:35.255 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:35.393 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:35.523 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:35.524 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:26:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:35.524 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:35.524 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T09:26:35.524 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 765 k 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 771 k 2024-09-11T09:26:35.525 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:35.637 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:35.799 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-11T09:26:35.800 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:35.800 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-11T09:26:35.800 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:26:35.805 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:26:35.805 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:26:35.811 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:26:35.812 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:26:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:35.886 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:35.886 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:35.886 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:35.887 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:26:35.948 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T09:26:35.977 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-11T09:26:35.978 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:35.978 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T09:26:35.978 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T09:26:35.978 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T09:26:35.978 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:36.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T09:26:36.416 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:36.440 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T09:26:36.466 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-11T09:26:36.466 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:36.466 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T09:26:36.466 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T09:26:36.466 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T09:26:36.466 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:36.487 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:36.642 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T09:26:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-11T09:26:36.725 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:36.725 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T09:26:36.725 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T09:26:36.725 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T09:26:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:36.776 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:26:36.777 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 145 k 2024-09-11T09:26:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 435 k 2024-09-11T09:26:36.777 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:36.791 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T09:26:36.890 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:36.975 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 735 kB/s | 145 kB 00:00 2024-09-11T09:26:36.975 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:36.976 INFO:teuthology.orchestra.run.smithi096.stdout:Total 729 kB/s | 145 kB 00:00 2024-09-11T09:26:36.976 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:26:36.983 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:26:36.983 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:26:37.039 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:26:37.039 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:26:37.170 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:37.170 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:37.170 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:37.170 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:37.170 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:37.170 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:26:37.304 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:37.329 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:37.329 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T09:26:37.329 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:37.404 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T09:26:37.969 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:38.102 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:38.102 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:38.102 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 145 k 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 435 k 2024-09-11T09:26:38.103 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:38.303 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:38.303 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:38.306 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 714 kB/s | 145 kB 00:00 2024-09-11T09:26:38.307 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:38.308 INFO:teuthology.orchestra.run.smithi107.stdout:Total 708 kB/s | 145 kB 00:00 2024-09-11T09:26:38.308 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:26:38.315 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:26:38.315 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:26:38.372 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:26:38.372 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-11T09:26:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-11T09:26:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-11T09:26:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-11T09:26:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-11T09:26:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-11T09:26:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-11T09:26:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-11T09:26:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-11T09:26:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-11T09:26:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-11T09:26:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-11T09:26:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-11T09:26:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-11T09:26:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-11T09:26:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-11T09:26:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-11T09:26:38.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr 2024-09-11T09:26:38.534 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:26:38.616 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:38.640 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:38.641 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:38.641 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T09:26:38.641 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:39.106 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:39.197 INFO:teuthology.orchestra.run.smithi096.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:26:39.238 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:39.240 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:26:39.240 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:39.305 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:39.480 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T09:26:39.481 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T09:26:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:39.483 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:39.537 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:39.538 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:39.538 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:39.538 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:39.751 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-09-11T09:26:39.824 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-base 2024-09-11T09:26:39.864 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:40.000 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:40.000 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:26:40.000 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:40.000 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:40.000 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T09:26:40.001 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:40.002 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:40.002 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:40.002 INFO:teuthology.orchestra.run.smithi096.stdout:Install 12 Packages 2024-09-11T09:26:40.002 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:40.002 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 8.7 M 2024-09-11T09:26:40.002 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 104 M 2024-09-11T09:26:40.003 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:40.323 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:40.415 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:40.418 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:26:40.429 INFO:teuthology.orchestra.run.smithi096.stdout:(1/12): ceph-grafana-dashboards-19.1.1-298.gb53 127 kB/s | 32 kB 00:00 2024-09-11T09:26:40.454 INFO:teuthology.orchestra.run.smithi096.stdout:(2/12): ceph-prometheus-alerts-19.1.1-298.gb53b 60 kB/s | 16 kB 00:00 2024-09-11T09:26:40.460 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:40.461 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:26:40.461 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:40.509 INFO:teuthology.orchestra.run.smithi017.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:26:40.531 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T09:26:40.553 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:40.554 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:26:40.554 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:40.624 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install cephadm 2024-09-11T09:26:41.092 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:41.210 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:41.223 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T09:26:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T09:26:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:41.228 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-09-11T09:26:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:41.229 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 8.7 M 2024-09-11T09:26:41.229 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 104 M 2024-09-11T09:26:41.229 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:41.338 INFO:teuthology.orchestra.run.smithi096.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 305 kB/s | 269 kB 00:00 2024-09-11T09:26:41.349 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:41.349 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:26:41.349 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:41.349 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:26:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 765 k 2024-09-11T09:26:41.350 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 771 k 2024-09-11T09:26:41.351 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:26:41.522 INFO:teuthology.orchestra.run.smithi096.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.6 MB/s | 863 kB 00:00 2024-09-11T09:26:41.605 INFO:teuthology.orchestra.run.smithi096.stdout:(5/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3e 2.5 MB/s | 3.6 MB 00:01 2024-09-11T09:26:41.639 INFO:teuthology.orchestra.run.smithi096.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-09-11T09:26:41.664 INFO:teuthology.orchestra.run.smithi096.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-09-11T09:26:41.670 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): ceph-grafana-dashboards-19.1.1-298.gb53 101 kB/s | 32 kB 00:00 2024-09-11T09:26:41.695 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): ceph-prometheus-alerts-19.1.1-298.gb53b 48 kB/s | 16 kB 00:00 2024-09-11T09:26:41.697 INFO:teuthology.orchestra.run.smithi096.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-11T09:26:41.723 INFO:teuthology.orchestra.run.smithi096.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-11T09:26:41.757 INFO:teuthology.orchestra.run.smithi096.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-11T09:26:41.807 INFO:teuthology.orchestra.run.smithi096.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-11T09:26:41.909 INFO:teuthology.orchestra.run.smithi017.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 1.3 MB/s | 765 kB 00:00 2024-09-11T09:26:41.910 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:41.910 INFO:teuthology.orchestra.run.smithi017.stdout:Total 1.3 MB/s | 765 kB 00:00 2024-09-11T09:26:41.910 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:26:41.912 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-11T09:26:41.916 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:26:41.917 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:26:41.922 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:26:41.923 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:26:41.999 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:26:42.096 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.4 MB/s | 1.0 MB 00:00 2024-09-11T09:26:42.146 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-11T09:26:42.171 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-11T09:26:42.238 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-11T09:26:42.263 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-11T09:26:42.289 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-11T09:26:42.322 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-11T09:26:42.398 INFO:teuthology.orchestra.run.smithi096.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 525 kB/s | 1.0 MB 00:01 2024-09-11T09:26:42.404 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:42.405 INFO:teuthology.orchestra.run.smithi096.stdout:Total 3.6 MB/s | 8.7 MB 00:02 2024-09-11T09:26:42.431 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3 3.3 MB/s | 3.6 MB 00:01 2024-09-11T09:26:42.457 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:26:42.459 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:42.474 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:26:42.475 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:26:42.559 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:42.694 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:26:42.695 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:26:42.916 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:42.956 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 827 kB/s | 863 kB 00:01 2024-09-11T09:26:42.960 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:42.960 INFO:teuthology.orchestra.run.smithi107.stdout:Total 5.0 MB/s | 8.7 MB 00:01 2024-09-11T09:26:43.009 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:26:43.027 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:26:43.027 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:26:43.173 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:26:43.173 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:43.173 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:26:43.173 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:43.173 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:43.174 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:26:43.233 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:26:43.233 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:26:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-11T09:26:43.351 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-11T09:26:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-11T09:26:43.401 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T09:26:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-11T09:26:43.532 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-11T09:26:43.662 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-11T09:26:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-11T09:26:43.769 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:26:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-11T09:26:43.836 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-11T09:26:43.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-11T09:26:43.868 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-11T09:26:43.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-11T09:26:43.946 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-11T09:26:43.986 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-11T09:26:43.989 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:44.091 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-11T09:26:44.128 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:26:44.129 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:44.130 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:26:44.130 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:44.130 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 145 k 2024-09-11T09:26:44.130 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 435 k 2024-09-11T09:26:44.130 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:26:44.230 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-11T09:26:44.325 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-11T09:26:44.368 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-11T09:26:44.397 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-11T09:26:44.428 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-11T09:26:44.501 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 392 kB/s | 145 kB 00:00 2024-09-11T09:26:44.501 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:44.502 INFO:teuthology.orchestra.run.smithi017.stdout:Total 390 kB/s | 145 kB 00:00 2024-09-11T09:26:44.502 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:26:44.510 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:26:44.510 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:26:44.571 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:26:44.571 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:26:44.729 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:26:44.802 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:44.828 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:26:44.828 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T09:26:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-11T09:26:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T09:26:45.499 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-11T09:26:45.537 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T09:26:45.798 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-11T09:26:45.798 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:45.798 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:26:45.798 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:26:45.798 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:45.798 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:46.005 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr 2024-09-11T09:26:46.364 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T09:26:46.364 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-11T09:26:46.365 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-11T09:26:46.592 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:59 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-11T09:26:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:46.675 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T09:26:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T09:26:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:46.677 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:46.686 INFO:teuthology.orchestra.run.smithi017.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:26:46.731 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:46.732 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:26:46.732 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:46.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T09:26:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T09:26:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-11T09:26:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-11T09:26:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-11T09:26:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-11T09:26:46.894 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-11T09:26:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-11T09:26:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-11T09:26:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-11T09:26:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-11T09:26:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-11T09:26:46.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-11T09:26:46.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T09:26:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-11T09:26:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:47.219 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:47.219 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T09:26:47.220 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T09:26:47.221 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:47.221 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:47.380 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:47.487 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T09:26:47.500 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:08 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:47.520 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T09:26:47.522 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout:Install 12 Packages 2024-09-11T09:26:47.523 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:47.524 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 8.7 M 2024-09-11T09:26:47.524 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 104 M 2024-09-11T09:26:47.524 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:26:47.641 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:47.642 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-11T09:26:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:26:47.642 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-11T09:26:47.642 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T09:26:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout:Install 12 Packages 2024-09-11T09:26:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:47.645 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 38 M 2024-09-11T09:26:47.645 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 212 M 2024-09-11T09:26:47.645 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:47.883 INFO:teuthology.orchestra.run.smithi096.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-11T09:26:47.925 INFO:teuthology.orchestra.run.smithi096.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-11T09:26:48.044 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:05 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:48.050 INFO:teuthology.orchestra.run.smithi096.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-09-11T09:26:48.100 INFO:teuthology.orchestra.run.smithi096.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-11T09:26:48.159 INFO:teuthology.orchestra.run.smithi096.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-11T09:26:48.177 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T09:26:48.179 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout:====================================================================================== 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-09-11T09:26:48.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:48.181 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 38 M 2024-09-11T09:26:48.181 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 212 M 2024-09-11T09:26:48.181 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:48.343 INFO:teuthology.orchestra.run.smithi096.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.2 MB/s | 3.0 MB 00:00 2024-09-11T09:26:48.421 INFO:teuthology.orchestra.run.smithi017.stdout:(1/12): ceph-grafana-dashboards-19.1.1-298.gb53 101 kB/s | 32 kB 00:00 2024-09-11T09:26:48.427 INFO:teuthology.orchestra.run.smithi096.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-11T09:26:48.437 INFO:teuthology.orchestra.run.smithi017.stdout:(2/12): ceph-prometheus-alerts-19.1.1-298.gb53b 49 kB/s | 16 kB 00:00 2024-09-11T09:26:48.484 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-11T09:26:48.501 INFO:teuthology.orchestra.run.smithi096.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-11T09:26:48.534 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-11T09:26:48.576 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 358 kB/s | 33 kB 00:00 2024-09-11T09:26:48.580 INFO:teuthology.orchestra.run.smithi017.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-11T09:26:48.593 INFO:teuthology.orchestra.run.smithi096.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-09-11T09:26:48.630 INFO:teuthology.orchestra.run.smithi017.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-11T09:26:48.643 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 272 kB/s | 18 kB 00:00 2024-09-11T09:26:48.680 INFO:teuthology.orchestra.run.smithi017.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-11T09:26:48.694 INFO:teuthology.orchestra.run.smithi096.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-11T09:26:48.705 INFO:teuthology.orchestra.run.smithi017.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 776 kB/s | 19 kB 00:00 2024-09-11T09:26:48.710 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 577 kB/s | 38 kB 00:00 2024-09-11T09:26:48.789 INFO:teuthology.orchestra.run.smithi017.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 24 MB/s | 2.0 MB 00:00 2024-09-11T09:26:48.802 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-11T09:26:48.815 INFO:teuthology.orchestra.run.smithi017.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-11T09:26:48.865 INFO:teuthology.orchestra.run.smithi017.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 614 kB/s | 31 kB 00:00 2024-09-11T09:26:48.868 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-11T09:26:48.898 INFO:teuthology.orchestra.run.smithi017.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-11T09:26:48.944 INFO:teuthology.orchestra.run.smithi096.stdout:(11/12): ceph-mgr-diskprediction-local-19.1.1-2 6.0 MB/s | 7.4 MB 00:01 2024-09-11T09:26:48.952 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 20 MB/s | 4.9 MB 00:00 2024-09-11T09:26:49.053 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-09-11T09:26:49.078 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-11T09:26:49.378 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 50 MB/s | 16 MB 00:00 2024-09-11T09:26:49.424 INFO:teuthology.orchestra.run.smithi017.stdout:(11/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3 2.7 MB/s | 3.6 MB 00:01 2024-09-11T09:26:49.795 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): ceph-mgr-diskprediction-local-19.1.1-2 5.0 MB/s | 7.4 MB 00:01 2024-09-11T09:26:49.796 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:49.796 INFO:teuthology.orchestra.run.smithi107.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-11T09:26:49.878 INFO:teuthology.orchestra.run.smithi096.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-11T09:26:49.879 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:49.879 INFO:teuthology.orchestra.run.smithi096.stdout:Total 17 MB/s | 38 MB 00:02 2024-09-11T09:26:50.153 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:26:50.178 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:26:50.178 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:26:50.216 INFO:teuthology.orchestra.run.smithi017.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 528 kB/s | 863 kB 00:01 2024-09-11T09:26:50.221 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:26:50.221 INFO:teuthology.orchestra.run.smithi017.stdout:Total 3.2 MB/s | 8.7 MB 00:02 2024-09-11T09:26:50.233 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:26:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:26:50.260 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:26:50.276 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:26:50.295 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:26:50.296 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:26:50.516 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:26:50.517 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:26:50.598 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:26:50.599 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:26:50.682 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:26:50.682 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:26:51.110 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:26:51.184 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-11T09:26:51.219 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-11T09:26:51.267 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-11T09:26:51.304 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-11T09:26:51.355 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:26:51.415 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-11T09:26:51.421 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T09:26:51.460 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T09:26:51.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T09:26:51.562 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-11T09:26:51.565 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:26:51.603 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T09:26:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T09:26:51.653 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-11T09:26:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T09:26:51.701 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-11T09:26:51.737 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-11T09:26:51.769 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-11T09:26:51.784 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T09:26:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T09:26:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T09:26:51.819 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T09:26:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T09:26:52.003 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T09:26:52.051 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T09:26:52.137 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T09:26:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T09:26:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T09:26:52.795 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T09:26:52.887 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-11T09:26:52.919 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T09:26:52.923 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T09:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T09:26:54.185 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T09:26:54.192 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-11T09:26:54.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-11T09:26:54.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-11T09:26:54.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-11T09:26:54.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-11T09:26:54.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-11T09:26:54.513 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-11T09:26:54.513 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T09:26:54.514 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T09:26:54.515 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T09:26:54.515 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:54.515 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:26:54.724 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T09:26:55.017 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T09:26:55.045 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T09:26:55.219 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T09:26:55.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T09:26:55.309 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:08 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:26:55.450 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:26:55.451 INFO:teuthology.orchestra.run.smithi017.stdout:====================================================================================== 2024-09-11T09:26:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:26:55.451 INFO:teuthology.orchestra.run.smithi017.stdout:====================================================================================== 2024-09-11T09:26:55.451 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:26:55.451 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T09:26:55.452 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T09:26:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T09:26:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:55.453 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:26:55.453 INFO:teuthology.orchestra.run.smithi017.stdout:====================================================================================== 2024-09-11T09:26:55.453 INFO:teuthology.orchestra.run.smithi017.stdout:Install 12 Packages 2024-09-11T09:26:55.453 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:26:55.454 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 38 M 2024-09-11T09:26:55.454 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 212 M 2024-09-11T09:26:55.454 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:26:56.182 INFO:teuthology.orchestra.run.smithi017.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 859 kB/s | 191 kB 00:00 2024-09-11T09:26:56.249 INFO:teuthology.orchestra.run.smithi017.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-11T09:26:56.316 INFO:teuthology.orchestra.run.smithi017.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-11T09:26:56.383 INFO:teuthology.orchestra.run.smithi017.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 273 kB/s | 18 kB 00:00 2024-09-11T09:26:56.433 INFO:teuthology.orchestra.run.smithi017.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-11T09:26:56.550 INFO:teuthology.orchestra.run.smithi017.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-29 13 MB/s | 7.4 MB 00:00 2024-09-11T09:26:56.700 INFO:teuthology.orchestra.run.smithi017.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-11T09:26:56.976 INFO:teuthology.orchestra.run.smithi017.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.1 MB/s | 3.0 MB 00:00 2024-09-11T09:26:57.084 INFO:teuthology.orchestra.run.smithi017.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T09:26:57.624 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T09:26:57.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T09:26:57.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T09:26:57.625 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T09:26:57.776 INFO:teuthology.orchestra.run.smithi017.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.7 MB/s | 4.9 MB 00:01 2024-09-11T09:26:57.890 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T09:26:57.890 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-11T09:26:57.890 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T09:26:57.890 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T09:26:57.890 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T09:26:57.891 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T09:26:57.985 INFO:teuthology.orchestra.run.smithi017.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-09-11T09:26:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T09:26:58.017 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T09:26:58.018 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T09:26:58.019 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:58.019 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:26:58.299 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T09:26:58.300 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:58.301 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:26:58.387 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-09-11T09:26:58.689 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-rook 2024-09-11T09:26:58.958 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:15 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:26:59.092 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T09:26:59.093 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:26:59.094 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:26:59.095 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-09-11T09:26:59.095 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:26:59.095 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.2 M 2024-09-11T09:26:59.095 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-09-11T09:26:59.095 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:26:59.258 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:20 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:26:59.391 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:26:59.392 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:59.392 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:26:59.392 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:59.392 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T09:26:59.393 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout:Install 13 Packages 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 2.2 M 2024-09-11T09:26:59.394 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 26 M 2024-09-11T09:26:59.395 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:26:59.463 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 293 kB/s | 49 kB 00:00 2024-09-11T09:26:59.588 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 87 kB/s | 26 kB 00:00 2024-09-11T09:26:59.614 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 60 kB/s | 19 kB 00:00 2024-09-11T09:26:59.878 INFO:teuthology.orchestra.run.smithi096.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 270 kB/s | 26 kB 00:00 2024-09-11T09:26:59.902 INFO:teuthology.orchestra.run.smithi096.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 159 kB/s | 19 kB 00:00 2024-09-11T09:26:59.989 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 423 kB/s | 159 kB 00:00 2024-09-11T09:26:59.995 INFO:teuthology.orchestra.run.smithi096.stdout:(3/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 232 kB/s | 49 kB 00:00 2024-09-11T09:27:00.014 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-11T09:27:00.037 INFO:teuthology.orchestra.run.smithi096.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-11T09:27:00.056 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 775 kB/s | 32 kB 00:00 2024-09-11T09:27:00.062 INFO:teuthology.orchestra.run.smithi096.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-11T09:27:00.082 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 596 kB/s | 279 kB 00:00 2024-09-11T09:27:00.095 INFO:teuthology.orchestra.run.smithi096.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-09-11T09:27:00.107 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-11T09:27:00.129 INFO:teuthology.orchestra.run.smithi096.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-11T09:27:00.140 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 355 kB/s | 54 kB 00:00 2024-09-11T09:27:00.154 INFO:teuthology.orchestra.run.smithi096.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-11T09:27:00.180 INFO:teuthology.orchestra.run.smithi096.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-11T09:27:00.182 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-11T09:27:00.205 INFO:teuthology.orchestra.run.smithi096.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T09:27:00.216 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 801 kB/s | 59 kB 00:00 2024-09-11T09:27:00.230 INFO:teuthology.orchestra.run.smithi096.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-11T09:27:00.241 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-11T09:27:00.255 INFO:teuthology.orchestra.run.smithi096.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-11T09:27:00.283 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.0 MB/s | 1.0 MB 00:00 2024-09-11T09:27:00.286 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:00.286 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-09-11T09:27:00.297 INFO:teuthology.orchestra.run.smithi096.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-11T09:27:00.299 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:00.300 INFO:teuthology.orchestra.run.smithi096.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-11T09:27:00.349 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:00.364 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:00.365 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:00.366 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:00.382 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:00.382 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:00.454 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:00.457 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:00.470 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:00.470 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:00.963 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:00.981 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:01.130 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T09:27:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T09:27:01.201 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T09:27:01.206 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T09:27:01.257 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T09:27:01.263 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T09:27:01.295 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T09:27:01.300 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T09:27:01.341 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T09:27:01.344 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T09:27:01.443 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T09:27:01.454 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T09:27:01.501 INFO:teuthology.orchestra.run.smithi017.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.7 MB/s | 16 MB 00:04 2024-09-11T09:27:01.502 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:01.503 INFO:teuthology.orchestra.run.smithi017.stdout:Total 6.3 MB/s | 38 MB 00:06 2024-09-11T09:27:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T09:27:01.556 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T09:27:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T09:27:01.613 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T09:27:01.872 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:01.899 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:01.899 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:02.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T09:27:02.330 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T09:27:02.345 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:02.346 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:02.374 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T09:27:02.393 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T09:27:02.408 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T09:27:02.427 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T09:27:02.456 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T09:27:02.475 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T09:27:02.486 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T09:27:02.506 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T09:27:03.139 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:03.215 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T09:27:03.259 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T09:27:03.283 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T09:27:03.283 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T09:27:03.284 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T09:27:03.286 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T09:27:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T09:27:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T09:27:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T09:27:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T09:27:03.287 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T09:27:03.365 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T09:27:03.401 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T09:27:03.595 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T09:27:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:03.614 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:03.624 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T09:27:03.632 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T09:27:03.633 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T09:27:03.634 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T09:27:03.634 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T09:27:03.634 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T09:27:03.634 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T09:27:03.634 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:03.634 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T09:27:03.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T09:27:03.901 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T09:27:04.392 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T09:27:04.472 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:25 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:04.475 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:21 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T09:27:04.607 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:04.607 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:04.608 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:04.609 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 7.0 M 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T09:27:04.610 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout:Install 6 Packages 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 7.0 M 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 33 M 2024-09-11T09:27:04.611 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:04.854 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-11T09:27:04.912 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-11T09:27:04.937 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-11T09:27:04.963 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 611 kB/s | 146 kB 00:00 2024-09-11T09:27:04.988 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-11T09:27:05.092 INFO:teuthology.orchestra.run.smithi096.stdout:(1/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 701 kB/s | 146 kB 00:00 2024-09-11T09:27:05.097 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T09:27:05.099 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:05.099 INFO:teuthology.orchestra.run.smithi107.stdout:Total 14 MB/s | 7.0 MB 00:00 2024-09-11T09:27:05.117 INFO:teuthology.orchestra.run.smithi096.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-09-11T09:27:05.167 INFO:teuthology.orchestra.run.smithi096.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-11T09:27:05.169 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:05.180 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:05.180 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:05.193 INFO:teuthology.orchestra.run.smithi096.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-11T09:27:05.226 INFO:teuthology.orchestra.run.smithi096.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-11T09:27:05.278 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:05.279 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:05.535 INFO:teuthology.orchestra.run.smithi096.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-09-11T09:27:05.537 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:05.537 INFO:teuthology.orchestra.run.smithi096.stdout:Total 7.6 MB/s | 7.0 MB 00:00 2024-09-11T09:27:05.584 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:05.607 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:05.619 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:05.619 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:05.718 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:05.719 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:05.743 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T09:27:05.807 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T09:27:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T09:27:06.025 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:06.212 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T09:27:06.270 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T09:27:06.324 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T09:27:06.409 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T09:27:06.473 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T09:27:06.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T09:27:06.803 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T09:27:06.890 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T09:27:06.933 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T09:27:06.954 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T09:27:06.957 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T09:27:06.982 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T09:27:07.270 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T09:27:07.270 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-11T09:27:07.270 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T09:27:07.270 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T09:27:07.270 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T09:27:07.270 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T09:27:07.556 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T09:27:07.557 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:07.557 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T09:27:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-11T09:27:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T09:27:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T09:27:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T09:27:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T09:27:07.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-09-11T09:27:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T09:27:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:08.044 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:08.044 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:08.045 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:08.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-fuse 2024-09-11T09:27:08.354 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:25 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:08.488 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:08.488 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:08.488 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-09-11T09:27:08.489 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:08.490 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 977 k 2024-09-11T09:27:08.490 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.9 M 2024-09-11T09:27:08.490 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:08.838 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:08.849 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 771 kB/s | 80 kB 00:00 2024-09-11T09:27:08.972 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:08.973 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:08.974 INFO:teuthology.orchestra.run.smithi096.stdout:Install 2 Packages 2024-09-11T09:27:08.974 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:08.974 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 977 k 2024-09-11T09:27:08.974 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 2.9 M 2024-09-11T09:27:08.974 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:09.049 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 2.9 MB/s | 897 kB 00:00 2024-09-11T09:27:09.050 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:09.050 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.7 MB/s | 977 kB 00:00 2024-09-11T09:27:09.062 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:09.085 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:09.085 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:09.166 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:09.166 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:09.361 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:09.449 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-11T09:27:09.449 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T09:27:09.450 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T09:27:09.496 INFO:teuthology.orchestra.run.smithi096.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 299 kB/s | 80 kB 00:00 2024-09-11T09:27:09.501 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T09:27:09.529 INFO:teuthology.orchestra.run.smithi096.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 2.9 MB/s | 897 kB 00:00 2024-09-11T09:27:09.529 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:09.530 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.7 MB/s | 977 kB 00:00 2024-09-11T09:27:09.542 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:09.565 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:09.565 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:09.573 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T09:27:09.646 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:09.646 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:09.846 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T09:27:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:09.890 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:09.990 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T09:27:10.062 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T09:27:10.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-rook 2024-09-11T09:27:10.280 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T09:27:10.280 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-11T09:27:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T09:27:10.550 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:10.551 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T09:27:10.551 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:10.551 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:10.762 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T09:27:10.762 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-11T09:27:10.821 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-volume 2024-09-11T09:27:10.870 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:23 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:11.011 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:11.012 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:11.012 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:27:11.012 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:11.012 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:11.012 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T09:27:11.012 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T09:27:11.013 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout:Install 13 Packages 2024-09-11T09:27:11.014 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:11.015 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 2.2 M 2024-09-11T09:27:11.015 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 26 M 2024-09-11T09:27:11.015 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:11.038 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T09:27:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:11.039 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T09:27:11.039 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:11.039 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:11.328 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-volume 2024-09-11T09:27:11.400 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:11.539 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:11.539 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 285 k 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 1.3 M 2024-09-11T09:27:11.540 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:11.734 INFO:teuthology.orchestra.run.smithi017.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 202 kB/s | 26 kB 00:00 2024-09-11T09:27:11.759 INFO:teuthology.orchestra.run.smithi017.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 126 kB/s | 19 kB 00:00 2024-09-11T09:27:11.781 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-volume-19.1.1-298.gb53b6d3e.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-11T09:27:11.781 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:11.782 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-11T09:27:11.782 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:11.784 INFO:teuthology.orchestra.run.smithi017.stdout:(3/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 278 kB/s | 49 kB 00:00 2024-09-11T09:27:11.790 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:11.790 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:11.824 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:11.825 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:11.902 INFO:teuthology.orchestra.run.smithi017.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-11T09:27:11.909 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:32 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:11.927 INFO:teuthology.orchestra.run.smithi017.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-09-11T09:27:11.953 INFO:teuthology.orchestra.run.smithi017.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-11T09:27:11.977 INFO:teuthology.orchestra.run.smithi017.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 643 kB/s | 32 kB 00:00 2024-09-11T09:27:12.003 INFO:teuthology.orchestra.run.smithi017.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 294 kB/s | 14 kB 00:00 2024-09-11T09:27:12.028 INFO:teuthology.orchestra.run.smithi017.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-11T09:27:12.044 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:12.044 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:12.045 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:12.045 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-11T09:27:12.045 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:12.045 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 285 k 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 1.3 M 2024-09-11T09:27:12.046 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:12.061 INFO:teuthology.orchestra.run.smithi017.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-11T09:27:12.087 INFO:teuthology.orchestra.run.smithi017.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-11T09:27:12.108 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:12.112 INFO:teuthology.orchestra.run.smithi017.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-09-11T09:27:12.124 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:12.124 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:27:12.124 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T09:27:12.124 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:12.145 INFO:teuthology.orchestra.run.smithi017.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-11T09:27:12.148 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:12.149 INFO:teuthology.orchestra.run.smithi017.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-11T09:27:12.217 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:12.234 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:12.234 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:12.276 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-volume-19.1.1-298.gb53b6d3e.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-11T09:27:12.277 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:12.277 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-11T09:27:12.277 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:12.285 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:12.285 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:12.313 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:12.313 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:12.333 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:12.333 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:12.526 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:12.595 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:12.611 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:12.611 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:27:12.611 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T09:27:12.611 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:12.849 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:13.027 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T09:27:13.083 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T09:27:13.096 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:13.096 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:13.096 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:13.096 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:13.096 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:13.096 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:13.138 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T09:27:13.173 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T09:27:13.216 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T09:27:13.322 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T09:27:13.433 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-09-11T09:27:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T09:27:13.501 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T09:27:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:13.665 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:13.666 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:13.666 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:13.666 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:13.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados-devel 2024-09-11T09:27:14.008 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:14.144 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:14.145 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:14.145 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-11T09:27:14.145 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 127 k 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 456 k 2024-09-11T09:27:14.146 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:14.259 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T09:27:14.315 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T09:27:14.348 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T09:27:14.363 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 584 kB/s | 127 kB 00:00 2024-09-11T09:27:14.363 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:14.364 INFO:teuthology.orchestra.run.smithi107.stdout:Total 580 kB/s | 127 kB 00:00 2024-09-11T09:27:14.364 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:14.369 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:14.369 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:14.406 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T09:27:14.433 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:14.433 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:14.433 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T09:27:14.536 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:14.593 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:14.671 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:14.672 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:14.673 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:14.673 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:27:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:14.673 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 127 k 2024-09-11T09:27:14.673 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 456 k 2024-09-11T09:27:14.673 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:14.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:14.866 INFO:teuthology.orchestra.run.smithi096.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 657 kB/s | 127 kB 00:00 2024-09-11T09:27:14.866 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:14.867 INFO:teuthology.orchestra.run.smithi096.stdout:Total 652 kB/s | 127 kB 00:00 2024-09-11T09:27:14.867 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:14.872 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:14.872 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:14.936 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:14.937 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:15.099 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:15.151 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:15.205 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:15.315 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-11T09:27:15.315 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-11T09:27:15.315 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T09:27:15.315 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T09:27:15.316 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T09:27:15.316 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T09:27:15.316 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T09:27:15.316 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T09:27:15.316 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T09:27:15.316 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T09:27:15.317 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T09:27:15.317 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T09:27:15.317 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T09:27:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:15.439 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:15.439 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:15.439 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:15.642 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:15.664 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T09:27:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:15.672 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:15.904 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:15.904 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:15.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T09:27:16.162 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs2 2024-09-11T09:27:16.243 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:16.334 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:16.378 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:16.379 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:16.379 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:16.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-09-11T09:27:16.542 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:16.683 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T09:27:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout:Install 6 Packages 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 7.0 M 2024-09-11T09:27:16.685 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 33 M 2024-09-11T09:27:16.686 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:16.737 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:16.831 INFO:teuthology.orchestra.run.smithi096.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:16.876 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:16.877 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:16.877 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:16.945 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install libcephfs-devel 2024-09-11T09:27:17.007 INFO:teuthology.orchestra.run.smithi017.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-11T09:27:17.032 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:17.032 INFO:teuthology.orchestra.run.smithi017.stdout:(2/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 622 kB/s | 146 kB 00:00 2024-09-11T09:27:17.082 INFO:teuthology.orchestra.run.smithi017.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T09:27:17.257 INFO:teuthology.orchestra.run.smithi017.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.3 MB/s | 548 kB 00:00 2024-09-11T09:27:17.257 INFO:teuthology.orchestra.run.smithi017.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-11T09:27:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:17.257 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:17.257 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 32 k 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 142 k 2024-09-11T09:27:17.258 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:17.326 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 199 kB/s | 32 kB 00:00 2024-09-11T09:27:17.326 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:17.326 INFO:teuthology.orchestra.run.smithi107.stdout:Total 197 kB/s | 32 kB 00:00 2024-09-11T09:27:17.327 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:17.330 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:17.330 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:17.344 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:17.344 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:17.432 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:17.523 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:17.591 INFO:teuthology.orchestra.run.smithi017.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.5 MB/s | 6.0 MB 00:00 2024-09-11T09:27:17.594 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:17.594 INFO:teuthology.orchestra.run.smithi017.stdout:Total 7.7 MB/s | 7.0 MB 00:00 2024-09-11T09:27:17.663 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:17.664 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:17.664 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-09-11T09:27:17.664 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 32 k 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 142 k 2024-09-11T09:27:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:17.667 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:17.680 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:17.680 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:17.786 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:17.786 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:17.828 INFO:teuthology.orchestra.run.smithi096.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 194 kB/s | 32 kB 00:00 2024-09-11T09:27:17.830 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:17.830 INFO:teuthology.orchestra.run.smithi096.stdout:Total 191 kB/s | 32 kB 00:00 2024-09-11T09:27:17.830 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:17.837 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:17.837 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:17.856 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:17.857 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:17.913 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:17.976 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:18.094 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:18.105 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:18.144 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:18.145 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:18.145 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:18.145 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:18.145 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:18.145 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:18.283 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T09:27:18.323 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-09-11T09:27:18.345 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T09:27:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:18.724 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:18.725 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:18.725 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:35 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:18.913 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T09:27:18.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librados2 2024-09-11T09:27:18.973 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:19.008 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T09:27:19.016 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:19.017 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:19.017 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:19.078 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-09-11T09:27:19.079 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T09:27:19.114 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T09:27:19.495 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:19.586 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:19.630 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:19.631 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:19.631 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:19.642 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:19.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install librbd1 2024-09-11T09:27:19.732 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:19.775 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:19.777 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:19.777 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:19.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-09-11T09:27:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-11T09:27:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-11T09:27:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T09:27:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T09:27:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T09:27:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T09:27:20.144 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:20.145 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:20.268 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:20.358 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:20.365 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-fuse 2024-09-11T09:27:20.397 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:20.401 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:20.403 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:20.403 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:20.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rados 2024-09-11T09:27:20.487 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:20.530 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:20.531 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:20.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-09-11T09:27:20.961 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:21.043 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:21.102 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T09:27:21.103 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:Install 2 Packages 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 977 k 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 2.9 M 2024-09-11T09:27:21.104 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:21.135 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:21.162 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:21.179 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:21.180 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:21.180 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:21.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rgw 2024-09-11T09:27:21.253 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:21.296 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:21.297 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:21.298 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:21.365 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-09-11T09:27:21.385 INFO:teuthology.orchestra.run.smithi017.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 490 kB/s | 80 kB 00:00 2024-09-11T09:27:21.518 INFO:teuthology.orchestra.run.smithi017.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 3.0 MB/s | 897 kB 00:00 2024-09-11T09:27:21.519 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:21.519 INFO:teuthology.orchestra.run.smithi017.stdout:Total 2.3 MB/s | 977 kB 00:00 2024-09-11T09:27:21.532 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:21.555 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:21.556 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:21.642 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:21.642 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:21.816 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:21.838 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:21.907 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:21.935 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:21.951 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:21.952 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:21.952 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T09:27:22.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-cephfs 2024-09-11T09:27:22.025 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:22.054 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T09:27:22.068 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:22.069 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:22.069 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:22.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-09-11T09:27:22.597 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:22.687 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:22.704 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:22.730 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:22.731 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:22.731 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-11T09:27:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-11T09:27:22.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install python3-rbd 2024-09-11T09:27:22.794 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:22.836 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:22.837 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:22.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-09-11T09:27:23.031 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T09:27:23.031 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:23.031 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:23.031 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T09:27:23.032 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:23.032 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:23.307 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-volume 2024-09-11T09:27:23.362 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:44 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:23.454 INFO:teuthology.orchestra.run.smithi096.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:23.486 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:23.496 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:23.497 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:23.497 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:23.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-fuse 2024-09-11T09:27:23.627 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:23.628 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:23.629 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:27:23.629 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:23.629 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 85 k 2024-09-11T09:27:23.629 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 231 k 2024-09-11T09:27:23.629 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 464 kB/s | 85 kB 00:00 2024-09-11T09:27:23.815 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:23.815 INFO:teuthology.orchestra.run.smithi107.stdout:Total 458 kB/s | 85 kB 00:00 2024-09-11T09:27:23.815 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:23.821 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:23.821 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:23.888 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:23.888 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:23.901 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:24.039 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:24.042 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:24.043 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:24.043 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2024-09-11T09:27:24.043 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:24.043 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:24.043 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T09:27:24.043 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 285 k 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 1.3 M 2024-09-11T09:27:24.044 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:24.137 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:24.140 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:24.274 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-volume-19.1.1-298.gb53b6d3e.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-11T09:27:24.274 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:24.275 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:24.275 INFO:teuthology.orchestra.run.smithi017.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-11T09:27:24.275 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:24.275 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:24.275 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 85 k 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 231 k 2024-09-11T09:27:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:24.284 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:24.284 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:24.320 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:24.320 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:24.473 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 432 kB/s | 85 kB 00:00 2024-09-11T09:27:24.474 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:24.474 INFO:teuthology.orchestra.run.smithi096.stdout:Total 429 kB/s | 85 kB 00:00 2024-09-11T09:27:24.474 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:24.481 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:24.481 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:24.544 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:24.544 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:24.551 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:24.552 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:24.617 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:24.635 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:24.635 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:27:24.635 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T09:27:24.635 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:24.702 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:24.804 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:24.819 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:24.819 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:24.820 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:24.820 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:25.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-09-11T09:27:25.259 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:25.500 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:25.501 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:25.501 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:25.501 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:25.646 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:25.660 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-11T09:27:25.660 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:25.660 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:25.660 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T09:27:25.660 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:25.661 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:25.776 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-mirror 2024-09-11T09:27:25.782 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:25.783 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:27:25.784 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:25.784 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 3.1 M 2024-09-11T09:27:25.784 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 13 M 2024-09-11T09:27:25.784 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:25.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install librados-devel 2024-09-11T09:27:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 9.3 MB/s | 3.1 MB 00:00 2024-09-11T09:27:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:Total 9.3 MB/s | 3.1 MB 00:00 2024-09-11T09:27:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:26.208 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:26.361 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:47 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:26.501 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:26.501 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repo Size 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:27:26.502 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:26.503 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 3.1 M 2024-09-11T09:27:26.503 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 13 M 2024-09-11T09:27:26.503 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:26.542 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:26.643 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:26.682 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:26.683 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:26.683 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2024-09-11T09:27:26.683 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:26.683 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:26.683 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 127 k 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 456 k 2024-09-11T09:27:26.684 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:26.719 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:26.744 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:26.745 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:27:26.745 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T09:27:26.745 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T09:27:26.745 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T09:27:26.745 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:26.896 INFO:teuthology.orchestra.run.smithi017.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 598 kB/s | 127 kB 00:00 2024-09-11T09:27:26.896 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:26.896 INFO:teuthology.orchestra.run.smithi017.stdout:Total 594 kB/s | 127 kB 00:00 2024-09-11T09:27:26.897 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:26.902 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:26.902 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:26.970 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:26.970 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:27.128 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:27.240 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:27.853 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:27.854 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:27.854 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:27.854 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:27.900 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:27.900 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:28.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-09-11T09:27:28.114 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install libcephfs2 2024-09-11T09:27:28.685 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:28.710 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:28.805 INFO:teuthology.orchestra.run.smithi017.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:28.820 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:28.820 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:28.820 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 171 k 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 494 k 2024-09-11T09:27:28.821 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-11T09:27:28.850 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:28.852 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:28.852 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:28.908 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install libcephfs-devel 2024-09-11T09:27:29.019 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 1.2 MB/s | 3.1 MB 00:02 2024-09-11T09:27:29.019 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:29.020 INFO:teuthology.orchestra.run.smithi096.stdout:Total 1.2 MB/s | 3.1 MB 00:02 2024-09-11T09:27:29.020 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:29.031 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:29.031 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:29.039 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 784 kB/s | 171 kB 00:00 2024-09-11T09:27:29.039 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:29.040 INFO:teuthology.orchestra.run.smithi107.stdout:Total 780 kB/s | 171 kB 00:00 2024-09-11T09:27:29.040 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-11T09:27:29.047 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-11T09:27:29.048 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-11T09:27:29.111 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-11T09:27:29.111 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-11T09:27:29.116 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:29.116 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:29.261 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-11T09:27:29.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:42 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:29.542 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:29.603 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:29.629 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:27:29.629 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T09:27:29.629 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T09:27:29.629 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T09:27:29.629 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:29.636 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:27:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:29.638 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 32 k 2024-09-11T09:27:29.638 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 142 k 2024-09-11T09:27:29.638 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:29.783 INFO:teuthology.orchestra.run.smithi017.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 220 kB/s | 32 kB 00:00 2024-09-11T09:27:29.783 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:29.783 INFO:teuthology.orchestra.run.smithi017.stdout:Total 217 kB/s | 32 kB 00:00 2024-09-11T09:27:29.784 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:29.787 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:29.787 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:29.801 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:29.802 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:29.878 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:29.897 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:30.009 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:30.113 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:30.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:30.113 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-11T09:27:30.114 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:30.114 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:30.114 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:30.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-09-11T09:27:30.368 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:30.608 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:30.608 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:30.608 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:30.608 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:30.608 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:30.609 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:30.767 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:30.767 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:30.768 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:30.768 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:30.768 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:30.768 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:30.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install librados2 2024-09-11T09:27:30.936 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:47 ago on Wed 11 Sep 2024 09:25:43 AM UTC. 2024-09-11T09:27:31.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-nbd 2024-09-11T09:27:31.029 INFO:teuthology.orchestra.run.smithi107.stdout:Package rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:31.074 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-11T09:27:31.075 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-11T09:27:31.075 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-11T09:27:31.138 DEBUG:teuthology.parallel:result is None 2024-09-11T09:27:31.376 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:44 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:31.470 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:31.516 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:31.517 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:31.517 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:31.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install librbd1 2024-09-11T09:27:31.605 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:52 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-09-11T09:27:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:31.740 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-11T09:27:31.740 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout:Install 1 Package 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 171 k 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 494 k 2024-09-11T09:27:31.741 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-11T09:27:31.956 INFO:teuthology.orchestra.run.smithi096.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 794 kB/s | 171 kB 00:00 2024-09-11T09:27:31.957 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:31.957 INFO:teuthology.orchestra.run.smithi096.stdout:Total 789 kB/s | 171 kB 00:00 2024-09-11T09:27:31.957 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-11T09:27:31.964 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-11T09:27:31.964 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-11T09:27:32.028 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-11T09:27:32.028 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-11T09:27:32.167 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:32.185 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-11T09:27:32.259 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:32.295 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:32.305 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:32.306 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:32.306 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:32.368 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-rados 2024-09-11T09:27:32.736 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:32.958 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:45 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:33.011 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:33.011 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:33.011 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-11T09:27:33.011 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:33.011 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:33.011 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:33.054 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:33.100 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:33.100 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:33.101 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:33.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-rgw 2024-09-11T09:27:33.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install rbd-nbd 2024-09-11T09:27:33.768 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:33.844 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:01:54 ago on Wed 11 Sep 2024 09:25:39 AM UTC. 2024-09-11T09:27:33.862 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:33.909 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:33.910 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:33.910 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:33.936 INFO:teuthology.orchestra.run.smithi096.stdout:Package rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:33.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-cephfs 2024-09-11T09:27:33.980 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-11T09:27:33.981 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-09-11T09:27:33.982 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-11T09:27:34.053 DEBUG:teuthology.parallel:result is None 2024-09-11T09:27:34.569 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:47 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:34.662 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:34.708 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:34.709 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:34.709 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:34.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install python3-rbd 2024-09-11T09:27:35.369 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:48 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:35.463 INFO:teuthology.orchestra.run.smithi017.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:35.509 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:35.510 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:35.510 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:35.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-fuse 2024-09-11T09:27:36.168 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:49 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:36.307 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:36.308 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:36.308 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2024-09-11T09:27:36.308 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 85 k 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 231 k 2024-09-11T09:27:36.309 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:36.519 INFO:teuthology.orchestra.run.smithi017.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 405 kB/s | 85 kB 00:00 2024-09-11T09:27:36.520 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:36.520 INFO:teuthology.orchestra.run.smithi017.stdout:Total 403 kB/s | 85 kB 00:00 2024-09-11T09:27:36.520 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:36.528 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:36.528 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:36.595 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:36.596 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:36.746 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:36.861 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:37.350 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:37.594 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:37.594 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:37.594 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:37.594 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:37.595 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:37.595 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:37.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-mirror 2024-09-11T09:27:38.474 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:51 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:38.619 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:38.620 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:38.620 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repo Size 2024-09-11T09:27:38.620 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:38.620 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 3.1 M 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 13 M 2024-09-11T09:27:38.621 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:38.932 INFO:teuthology.orchestra.run.smithi017.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 10 MB/s | 3.1 MB 00:00 2024-09-11T09:27:38.932 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:38.933 INFO:teuthology.orchestra.run.smithi017.stdout:Total 9.9 MB/s | 3.1 MB 00:00 2024-09-11T09:27:38.933 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:38.942 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:38.943 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:39.029 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:39.029 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:39.482 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:39.558 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:39.586 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:39.586 INFO:teuthology.orchestra.run.smithi017.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T09:27:39.586 INFO:teuthology.orchestra.run.smithi017.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T09:27:39.586 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T09:27:39.586 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T09:27:39.586 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:40.700 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:40.701 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:40.701 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:40.701 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:40.701 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:40.701 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:40.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-nbd 2024-09-11T09:27:41.525 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:54 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:41.665 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout:Install 1 Package 2024-09-11T09:27:41.666 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:41.667 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 171 k 2024-09-11T09:27:41.667 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 494 k 2024-09-11T09:27:41.667 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2024-09-11T09:27:41.877 INFO:teuthology.orchestra.run.smithi017.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 816 kB/s | 171 kB 00:00 2024-09-11T09:27:41.877 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2024-09-11T09:27:41.878 INFO:teuthology.orchestra.run.smithi017.stdout:Total 810 kB/s | 171 kB 00:00 2024-09-11T09:27:41.878 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2024-09-11T09:27:41.886 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2024-09-11T09:27:41.886 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2024-09-11T09:27:41.954 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2024-09-11T09:27:41.954 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2024-09-11T09:27:42.114 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2024-09-11T09:27:42.219 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:42.637 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:42.933 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-11T09:27:42.933 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:42.933 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2024-09-11T09:27:42.933 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T09:27:42.933 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:42.933 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:43.164 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install rbd-nbd 2024-09-11T09:27:43.760 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:56 ago on Wed 11 Sep 2024 09:25:47 AM UTC. 2024-09-11T09:27:43.855 INFO:teuthology.orchestra.run.smithi017.stdout:Package rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-11T09:27:43.902 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2024-09-11T09:27:43.903 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2024-09-11T09:27:43.903 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2024-09-11T09:27:43.967 DEBUG:teuthology.parallel:result is None 2024-09-11T09:27:43.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:27:44.107 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T09:27:44.129 INFO:teuthology.orchestra.run.smithi017.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T09:27:44.130 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T09:27:44.130 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T09:27:44.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:27:44.269 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T09:27:44.294 INFO:teuthology.orchestra.run.smithi096.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T09:27:44.294 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T09:27:44.295 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T09:27:44.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T09:27:44.436 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T09:27:44.462 INFO:teuthology.orchestra.run.smithi107.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T09:27:44.462 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T09:27:44.462 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T09:27:44.464 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T09:27:44.464 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:44.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T09:27:44.493 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:44.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T09:27:44.523 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:27:44.523 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T09:27:44.552 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T09:27:44.553 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:44.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T09:27:44.581 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T09:27:44.650 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:44.650 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T09:27:44.679 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T09:27:44.748 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:27:44.748 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T09:27:44.778 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T09:27:44.846 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T09:27:44.846 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:44.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T09:27:44.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T09:27:44.945 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:44.945 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T09:27:44.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T09:27:45.042 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:27:45.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T09:27:45.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T09:27:45.139 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T09:27:45.139 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:45.140 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T09:27:45.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T09:27:45.242 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:45.242 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T09:27:45.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T09:27:45.341 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:27:45.341 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T09:27:45.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T09:27:45.437 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T09:27:45.527 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T09:27:45.527 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2024-09-11T09:27:45.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-09-11T09:27:45.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2024-09-11T09:27:45.559 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T09:27:45.560 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T09:27:45.601 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T09:27:45.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T09:27:45.630 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T09:27:45.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T09:27:45.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T09:27:45.674 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T09:27:45.714 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T09:27:45.714 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-11T09:27:45.714 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900713', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 6175, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'sleep_before_teardown': 0, 'subset': '88/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_daaf78de334647fc0c800f4ac495cc9475a26bae/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae', 'targets': {'smithi017.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEKUc2QfOB6XuJyv1zZxWvtUzcJDsvpb2kFQwcW/1jYYrAgPUaaHFiSWNknMxowsuXgN+kdXDiGwm1Ow73CRekU=', 'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMF7fhZ/UN5Ejb5s5dMmizz+6O3picx+MvgVyUGKkvLzIs4g0Xe2vd+hUkcDXqkgjKFINOYDMrQGf09vRwrNq/E=', 'smithi107.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAhRXNKiFBS44HYqCY9UeX03Sa8Cfgpd/kISiOWbfEQrVI7XQKtdJkhmyzDaFLjNXOEYcjPNOxNpr2IZ+XY3Kk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_21:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T09:27:45.715 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T09:27:45.741 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T09:27:45.743 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T09:27:45.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T09:27:45.800 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T09:27:45.802 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T09:27:45.829 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:45.829 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T09:27:45.888 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T09:27:45.889 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-09-11T09:27:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T09:27:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:45.947 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-11T09:27:45.947 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:45.948 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:45.948 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-11 09:27:39.869253244 +0000 2024-09-11T09:27:45.948 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-11 09:25:03.044033031 +0000 2024-09-11T09:27:45.948 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-11 09:25:03.044033031 +0000 2024-09-11T09:27:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-11 09:25:03.044033031 +0000 2024-09-11T09:27:45.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T09:27:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-11T09:27:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-11T09:27:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000111962 s, 4.6 MB/s 2024-09-11T09:27:46.016 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T09:27:46.076 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-09-11T09:27:46.133 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T09:27:46.133 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:46.133 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-11T09:27:46.133 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:46.133 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:46.133 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-11 09:27:39.870253235 +0000 2024-09-11T09:27:46.134 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-11 09:25:03.362029924 +0000 2024-09-11T09:27:46.134 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-11 09:25:03.362029924 +0000 2024-09-11T09:27:46.134 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-11 09:25:03.362029924 +0000 2024-09-11T09:27:46.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T09:27:46.200 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-11T09:27:46.201 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-11T09:27:46.201 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 9.8152e-05 s, 5.2 MB/s 2024-09-11T09:27:46.202 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T09:27:46.261 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-11 09:27:39.870253235 +0000 2024-09-11T09:27:46.319 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-11 09:25:03.682026797 +0000 2024-09-11T09:27:46.320 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-11 09:25:03.682026797 +0000 2024-09-11T09:27:46.320 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-11 09:25:03.682026797 +0000 2024-09-11T09:27:46.320 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T09:27:46.385 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-11T09:27:46.386 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-11T09:27:46.386 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000112408 s, 4.6 MB/s 2024-09-11T09:27:46.387 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T09:27:46.446 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-09-11T09:27:46.504 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T09:27:46.504 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:46.504 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T09:27:46.504 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:46.505 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:46.505 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-11 09:27:39.870253235 +0000 2024-09-11T09:27:46.505 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-11 09:25:04.002023670 +0000 2024-09-11T09:27:46.505 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-11 09:25:04.002023670 +0000 2024-09-11T09:27:46.505 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-11 09:25:04.002023670 +0000 2024-09-11T09:27:46.505 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T09:27:46.571 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-11T09:27:46.571 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-11T09:27:46.571 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000111251 s, 4.6 MB/s 2024-09-11T09:27:46.572 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T09:27:46.630 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:46.631 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:46.631 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T09:27:46.650 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T09:27:46.650 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-11 09:27:29.891152064 +0000 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-11 09:24:56.844938195 +0000 2024-09-11T09:27:46.709 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-11 09:24:56.844938195 +0000 2024-09-11T09:27:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-11 09:24:56.844938195 +0000 2024-09-11T09:27:46.710 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T09:27:46.776 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-11T09:27:46.777 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-11T09:27:46.777 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000346489 s, 1.5 MB/s 2024-09-11T09:27:46.778 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T09:27:46.838 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-11T09:27:46.897 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T09:27:46.898 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:46.898 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-11T09:27:46.898 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:46.898 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:46.898 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-11 09:27:29.891152064 +0000 2024-09-11T09:27:46.899 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-11 09:24:57.173930806 +0000 2024-09-11T09:27:46.899 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-11 09:24:57.173930806 +0000 2024-09-11T09:27:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-11 09:24:57.173930806 +0000 2024-09-11T09:27:46.899 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T09:27:46.965 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-11T09:27:46.965 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-11T09:27:46.965 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00021219 s, 2.4 MB/s 2024-09-11T09:27:46.967 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T09:27:47.025 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-11 09:27:29.892152041 +0000 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-11 09:24:57.492923641 +0000 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-11 09:24:57.492923641 +0000 2024-09-11T09:27:47.083 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-11 09:24:57.492923641 +0000 2024-09-11T09:27:47.084 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T09:27:47.147 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-11T09:27:47.147 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-11T09:27:47.147 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000230716 s, 2.2 MB/s 2024-09-11T09:27:47.149 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T09:27:47.204 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-11T09:27:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T09:27:47.261 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T09:27:47.261 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T09:27:47.261 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T09:27:47.261 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T09:27:47.262 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-11 09:27:29.892152041 +0000 2024-09-11T09:27:47.262 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-11 09:24:57.824916185 +0000 2024-09-11T09:27:47.262 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-11 09:24:57.824916185 +0000 2024-09-11T09:27:47.262 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-11 09:24:57.824916185 +0000 2024-09-11T09:27:47.263 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T09:27:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-11T09:27:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-11T09:27:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00019629 s, 2.6 MB/s 2024-09-11T09:27:47.326 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T09:27:47.382 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:47.382 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-11T09:27:47.382 INFO:tasks.ceph:Generating config... 2024-09-11T09:27:47.384 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-11T09:27:47.384 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-11T09:27:47.384 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-11T09:27:47.384 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T09:27:47.385 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T09:27:47.385 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T09:27:47.385 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T09:27:47.385 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-11T09:27:47.385 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-11T09:27:47.386 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T09:27:47.386 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T09:27:47.387 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T09:27:47.444 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T09:27:47.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T09:27:47.496 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T09:27:47.523 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17'), ('mon.c', '[v2:172.21.15.17:3301,v1:172.21.15.17:6790]'), ('mon.b', '172.21.15.96')] 2024-09-11T09:27:47.524 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.96', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-11T09:27:47.524 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:47.525 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T09:27:47.580 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.17 --addv c '[v2:172.21.15.17:3301,v1:172.21.15.17:6790]' --add b 172.21.15.96 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:47.669 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:47.669 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid 23a8bf95-caba-4fea-a762-61040a1c1b77 2024-09-11T09:27:47.669 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-09-11T09:27:47.669 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:fsid 23a8bf95-caba-4fea-a762-61040a1c1b77 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2024-09-11T09:27:47.670391+0000 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-09-11T09:27:47.670391+0000 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 17 (quincy) 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:election_strategy: 1 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:1: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:2: [v2:172.21.15.17:3301/0,v1:172.21.15.17:6790/0] mon.c 2024-09-11T09:27:47.670 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T09:27:47.671 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T09:27:47.687 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 23a8bf95-caba-4fea-a762-61040a1c1b77... 2024-09-11T09:27:47.688 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T09:27:47.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T09:27:47.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T09:27:47.772 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2024-09-11T09:27:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T09:27:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T09:27:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: mon_allow_pool_size_one = true 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.96" 2024-09-11T09:27:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: mon client directed command retry = 5 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on pool no app = False 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: ms inject socket failures = 5000 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = 23a8bf95-caba-4fea-a762-61040a1c1b77 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: osd mclock profile = high_recovery_ops 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2024-09-11T09:27:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug trim objects = true 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2024-09-11T09:27:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression algorithm = lz4 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 1/20 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 1/20 2024-09-11T09:27:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 4/10 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: osd blocked scrub grace period = 3600 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: osd max markdown count = 1000 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2024-09-11T09:27:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-11T09:27:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T09:27:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: # 1m isn't quite enough 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: mon_down_mkfs_grace = 2m 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_filestore_osds = false 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2024-09-11T09:27:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2024-09-11T09:27:47.784 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2024-09-11T09:27:47.785 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T09:27:47.786 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2024-09-11T09:27:47.787 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.788 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.96" 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: mon client directed command retry = 5 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on pool no app = False 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: ms inject socket failures = 5000 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 23a8bf95-caba-4fea-a762-61040a1c1b77 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2024-09-11T09:27:47.789 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2024-09-11T09:27:47.790 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug trim objects = true 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2024-09-11T09:27:47.791 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore compression algorithm = lz4 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore compression mode = aggressive 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 1/20 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 1/20 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 4/10 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T09:27:47.792 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: osd blocked scrub grace period = 3600 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: osd max markdown count = 1000 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-09-11T09:27:47.793 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2024-09-11T09:27:47.794 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2024-09-11T09:27:47.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2024-09-11T09:27:47.796 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-09-11T09:27:47.797 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.798 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-09-11T09:27:47.799 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.800 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = "172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.96" 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: mon client directed command retry = 5 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on pool no app = False 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: ms inject socket failures = 5000 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = 23a8bf95-caba-4fea-a762-61040a1c1b77 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-09-11T09:27:47.801 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug trim objects = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-09-11T09:27:47.802 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-09-11T09:27:47.803 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression algorithm = lz4 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore compression mode = aggressive 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 1/20 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 1/20 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 4/10 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2024-09-11T09:27:47.804 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: osd blocked scrub grace period = 3600 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: osd max markdown count = 1000 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-11T09:27:47.805 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-11T09:27:47.806 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-11T09:27:47.807 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.c] 2024-09-11T09:27:47.808 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.b] 2024-09-11T09:27:47.810 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T09:27:47.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T09:27:47.905 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T09:27:47.905 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:47.905 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T09:27:47.921 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:47.921 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T09:27:47.977 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:27:47.977 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:47.977 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T09:27:47.977 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T09:27:48.053 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:27:48.053 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:48.108 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:27:48.108 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:48.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T09:27:48.109 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T09:27:48.142 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:27:48.142 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:48.197 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2024-09-11T09:27:48.198 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:27:48.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T09:27:48.198 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T09:27:48.238 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:27:48.238 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:48.295 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T09:27:48.295 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T09:27:48.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T09:27:48.361 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T09:27:48.364 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T09:27:48.364 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T09:27:48.364 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T09:27:48.425 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-11T09:27:48.439 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T09:27:48.439 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-11T09:27:48.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T09:27:48.465 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:48.466 INFO:tasks.ceph:role: osd.0 2024-09-11T09:27:48.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:27:48.466 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T09:27:48.531 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T09:27:48.532 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T09:27:48.564 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-11T09:27:48.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-11T09:27:48.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T09:27:48.637 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T09:27:48.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T09:27:48.778 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:48.778 INFO:tasks.ceph:role: osd.1 2024-09-11T09:27:48.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:27:48.778 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T09:27:48.844 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T09:27:48.844 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T09:27:48.845 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T09:27:48.877 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-11T09:27:48.878 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-11T09:27:48.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T09:27:48.948 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T09:27:49.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T09:27:49.085 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:49.085 INFO:tasks.ceph:role: osd.2 2024-09-11T09:27:49.085 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:27:49.086 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T09:27:49.150 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T09:27:49.151 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T09:27:49.183 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-11T09:27:49.185 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-11T09:27:49.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T09:27:49.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T09:27:49.325 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:49.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:49.420+0000 7f69db013640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T09:27:49.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:49.420+0000 7f69db013640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T09:27:49.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:49.420+0000 7f69db013640 -1 bdev(0x562cd04d5800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T09:27:49.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:49.420+0000 7f69db013640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T09:27:52.334 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T09:27:52.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:52.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:52.459+0000 7f14e0e16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T09:27:52.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:52.459+0000 7f14e0e16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T09:27:52.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:52.459+0000 7f14e0e16640 -1 bdev(0x5651fc88b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T09:27:52.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:52.459+0000 7f14e0e16640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T09:27:55.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T09:27:55.408 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:27:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:55.502+0000 7fc5cd345640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T09:27:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:55.502+0000 7fc5cd345640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T09:27:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:55.502+0000 7fc5cd345640 -1 bdev(0x55c53f6bf800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T09:27:55.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:27:55.502+0000 7fc5cd345640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T09:27:58.661 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T09:27:58.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-11T09:27:58.722 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:58.723 INFO:tasks.ceph:role: osd.3 2024-09-11T09:27:58.723 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:27:58.723 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T09:27:58.798 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T09:27:58.799 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T09:27:59.313 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-11T09:27:59.315 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-11T09:27:59.315 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-11T09:27:59.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-11T09:27:59.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-11T09:27:59.542 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T09:27:59.542 INFO:tasks.ceph:role: osd.4 2024-09-11T09:27:59.543 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:27:59.543 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T09:27:59.615 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T09:27:59.616 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T09:28:00.073 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-11T09:28:00.075 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-11T09:28:00.075 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-11T09:28:00.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-11T09:28:00.191 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-11T09:28:00.258 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-11T09:28:00.258 INFO:tasks.ceph:role: osd.5 2024-09-11T09:28:00.259 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:28:00.259 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T09:28:00.332 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T09:28:00.332 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T09:28:00.332 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T09:28:00.332 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T09:28:00.332 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T09:28:00.332 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-11T09:28:00.333 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T09:28:00.333 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T09:28:00.333 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T09:28:00.333 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T09:28:00.821 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-11T09:28:00.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-11T09:28:00.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-11T09:28:00.967 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-11T09:28:01.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:28:01.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:01.095+0000 7ff4c22d4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-11T09:28:01.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:01.098+0000 7ff4c22d4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-11T09:28:01.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:01.098+0000 7ff4c22d4640 -1 bdev(0x556d982a9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-11T09:28:01.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:01.098+0000 7ff4c22d4640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-11T09:28:04.503 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-11T09:28:04.533 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:28:04.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:04.624+0000 7f5ea0c11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-11T09:28:04.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:04.624+0000 7f5ea0c11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-11T09:28:04.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:04.624+0000 7f5ea0c11640 -1 bdev(0x556f561ab800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-11T09:28:04.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:04.624+0000 7f5ea0c11640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-11T09:28:07.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-11T09:28:07.576 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:28:07.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:07.666+0000 7f7cd0015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-11T09:28:07.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:07.666+0000 7f7cd0015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-11T09:28:07.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:07.666+0000 7f7cd0015640 -1 bdev(0x559e4b6b9800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-11T09:28:07.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-11T09:28:07.666+0000 7f7cd0015640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-11T09:28:11.070 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-11T09:28:11.099 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T09:28:11.099 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:28:11.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T09:28:11.131 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:28:11.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T09:28:11.202 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:28:11.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T09:28:11.273 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:11.273 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T09:28:11.302 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:11.302 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-11T09:28:11.371 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:11.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-11T09:28:11.440 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:11.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-11T09:28:11.509 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-11T09:28:11.509 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-11T09:28:11.529 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T09:28:11.529 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T09:28:11.531 DEBUG:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T09:28:11.558 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2024-09-11T09:28:11.558 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCVYuFm7EMXGRAAWRTxXStKFs/AIQLTw2bu2Q== 2024-09-11T09:28:11.558 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2024-09-11T09:28:11.558 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCYYuFmcJduGxAAbiyhsosupdMBpJevSIeQng== 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCbYuFmIV8BHhAAxsNDOJU/JQp8t0NpWDLj9w== 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCUYuFmtSeiFRAAjoms85rsYhqLtAlek0sxBw== 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQChYuFmfjG7BRAAwA8+GCskA5xWQ3ZkR9vntA== 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.4] 2024-09-11T09:28:11.559 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCkYuFmSJlGJRAAuRAV0tsDtV6IRuNE21ndPQ== 2024-09-11T09:28:11.560 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.5] 2024-09-11T09:28:11.560 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCnYuFmi43AJxAAjZXLagkvtyu9nAcE5JRXaQ== 2024-09-11T09:28:11.560 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-09-11T09:28:11.560 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQCUYuFmt6BrGRAAJQfR//iYvJIijs0gtMzSXA== 2024-09-11T09:28:11.574 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-09-11T09:28:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCVYuFm7EMXGRAAWRTxXStKFs/AIQLTw2bu2Q== 2024-09-11T09:28:11.574 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-09-11T09:28:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCYYuFmcJduGxAAbiyhsosupdMBpJevSIeQng== 2024-09-11T09:28:11.574 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-09-11T09:28:11.574 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCbYuFmIV8BHhAAxsNDOJU/JQp8t0NpWDLj9w== 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCUYuFmtSeiFRAAjoms85rsYhqLtAlek0sxBw== 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.3] 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQChYuFmfjG7BRAAwA8+GCskA5xWQ3ZkR9vntA== 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.4] 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCkYuFmSJlGJRAAuRAV0tsDtV6IRuNE21ndPQ== 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.5] 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCnYuFmi43AJxAAjZXLagkvtyu9nAcE5JRXaQ== 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-09-11T09:28:11.575 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQCUYuFmt6BrGRAAJQfR//iYvJIijs0gtMzSXA== 2024-09-11T09:28:11.576 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.672 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.725 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.727 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T09:28:11.781 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T09:28:11.832 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.887 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T09:28:11.990 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T09:28:11.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T09:28:12.047 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T09:28:12.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T09:28:12.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T09:28:12.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T09:28:12.194 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T09:28:12.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T09:28:12.357 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T09:28:12.383 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T09:28:12.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T09:28:12.506 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T09:28:12.530 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:28:12.533 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T09:28:12.585 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T09:28:12.585 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T09:28:12.586 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T09:28:12.589 INFO:tasks.ceph.mon.a:Started 2024-09-11T09:28:12.589 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T09:28:12.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T09:28:12.591 INFO:tasks.ceph.mon.c:Started 2024-09-11T09:28:12.591 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T09:28:12.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T09:28:12.628 INFO:tasks.ceph.mon.b:Started 2024-09-11T09:28:12.628 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T09:28:12.628 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T09:28:12.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T09:28:12.630 INFO:tasks.ceph.mgr.x:Started 2024-09-11T09:28:12.630 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T09:28:12.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T09:28:17.986 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-09-11T09:28:17.998 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T09:28:17.998 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T09:28:17.998 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:28:17.998 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T09:28:18.026 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:28:18.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T09:28:18.092 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-11T09:28:18.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T09:28:18.157 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:18.157 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-11T09:28:18.188 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:18.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-11T09:28:18.255 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-11T09:28:18.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-11T09:28:18.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 44ce15f5-731e-4d58-9dd7-bd8f186e5783 0 2024-09-11T09:28:18.548 INFO:teuthology.orchestra.run.smithi096.stdout:0 2024-09-11T09:28:18.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new dba82e38-a74b-4d42-bad4-9b1e013d4da3 1 2024-09-11T09:28:18.755 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-11T09:28:18.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 64a33918-9bb2-4d59-a66e-73da15d78cbd 2 2024-09-11T09:28:18.803 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:18.804+0000 7f7a87d1f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T09:28:18.846 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:18.846+0000 7f7a87d1f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T09:28:18.949 INFO:teuthology.orchestra.run.smithi096.stdout:2 2024-09-11T09:28:18.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new a4774bc9-54ea-4272-b6b4-a6dd00c4c930 3 2024-09-11T09:28:18.991 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:18.992+0000 7f7a87d1f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T09:28:19.045 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:19.045+0000 7f7a87d1f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T09:28:19.170 INFO:teuthology.orchestra.run.smithi096.stdout:3 2024-09-11T09:28:19.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new a3984c6d-7250-4e85-a032-1dc0315642d8 4 2024-09-11T09:28:19.217 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:19.217+0000 7f7a87d1f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T09:28:19.332 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:19.332+0000 7f7a87d1f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T09:28:19.361 INFO:teuthology.orchestra.run.smithi096.stdout:4 2024-09-11T09:28:19.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 0a40bbb7-9d37-4a1b-b525-86608a3eaaa2 5 2024-09-11T09:28:19.592 INFO:teuthology.orchestra.run.smithi096.stdout:5 2024-09-11T09:28:19.603 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T09:28:19.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T09:28:19.606 INFO:tasks.ceph.osd.0:Started 2024-09-11T09:28:19.606 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T09:28:19.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T09:28:19.608 INFO:tasks.ceph.osd.1:Started 2024-09-11T09:28:19.608 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T09:28:19.608 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T09:28:19.610 INFO:tasks.ceph.osd.2:Started 2024-09-11T09:28:19.611 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-11T09:28:19.611 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-11T09:28:19.613 INFO:tasks.ceph.osd.3:Started 2024-09-11T09:28:19.613 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-11T09:28:19.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-11T09:28:19.615 INFO:tasks.ceph.osd.4:Started 2024-09-11T09:28:19.615 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-11T09:28:19.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-11T09:28:19.618 INFO:tasks.ceph.osd.5:Started 2024-09-11T09:28:19.618 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T09:28:19.677 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:19.676+0000 7f7a87d1f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T09:28:19.739 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:19.739+0000 7f7a87d1f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T09:28:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:19.803 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:28:19.590242+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T09:28:19.590242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:28:19.814 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T09:28:19.814 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T09:28:19.886 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:19.886+0000 7f7a87d1f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T09:28:20.420 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T09:28:20.593 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-11T09:28:20.593 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":8,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:28:19.590242+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T09:28:19.590242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:28:20.602 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-11T09:28:20.822 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:20.822+0000 7f9d6681b640 -1 Falling back to public interface 2024-09-11T09:28:20.830 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:20.830+0000 7f1e00013640 -1 Falling back to public interface 2024-09-11T09:28:20.843 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:20.843+0000 7f7a87d1f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T09:28:20.854 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:20.854+0000 7f4f3c1ae640 -1 Falling back to public interface 2024-09-11T09:28:20.862 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:20.861+0000 7f0929e3e640 -1 Falling back to public interface 2024-09-11T09:28:20.895 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:20.895+0000 7f7a87d1f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T09:28:21.077 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:21.077+0000 7fb1a7444640 -1 Falling back to public interface 2024-09-11T09:28:21.255 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.255+0000 7f7a87d1f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T09:28:21.308 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:21.308+0000 7f6619a0c640 -1 Falling back to public interface 2024-09-11T09:28:21.340 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.340+0000 7f7a87d1f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T09:28:21.447 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.446+0000 7f7a87d1f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T09:28:21.492 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.491+0000 7f7a87d1f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T09:28:21.603 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.603+0000 7f7a87d1f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T09:28:21.657 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.656+0000 7f7a87d1f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T09:28:21.701 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.700+0000 7f7a87d1f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T09:28:21.813 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.809+0000 7f7a87d1f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T09:28:21.860 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.859+0000 7f7a87d1f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T09:28:21.912 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.911+0000 7f7a87d1f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T09:28:21.978 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:21.977+0000 7f7a87d1f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T09:28:22.022 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:22.021+0000 7f7a87d1f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T09:28:22.065 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:22.064+0000 7f7a87d1f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T09:28:22.711 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:22.710+0000 7f7a87d1f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T09:28:24.350 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:24.351+0000 7f1e00013640 -1 osd.0 0 log_to_monitors true 2024-09-11T09:28:24.899 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:24.899+0000 7f4f3c1ae640 -1 osd.2 0 log_to_monitors true 2024-09-11T09:28:24.900 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:24.899+0000 7f9d6681b640 -1 osd.3 0 log_to_monitors true 2024-09-11T09:28:24.927 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:24.928+0000 7f0929e3e640 -1 osd.1 0 log_to_monitors true 2024-09-11T09:28:25.109 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:25.108+0000 7f6619a0c640 -1 osd.4 0 log_to_monitors true 2024-09-11T09:28:25.143 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:25.142+0000 7fb1a7444640 -1 osd.5 0 log_to_monitors true 2024-09-11T09:28:25.724 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:25.723+0000 7f7a27d23640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T09:28:25.725 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:28:25.724+0000 7f7a27d23640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T09:28:26.142 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:26.142+0000 7f1dfb8f5640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T09:28:26.163 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:26.164+0000 7f1df3715640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:28:27.209 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T09:28:27.311 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:27.312+0000 7f4f3813d640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T09:28:27.330 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:27.330+0000 7f4f2ff5d640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:28:27.347 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:27.348+0000 7f0925dcd640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T09:28:27.355 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:27.354+0000 7fb1a33d3640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T09:28:27.358 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:27.359+0000 7f091dbed640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:28:27.370 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:27.370+0000 7fb19b1f3640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:28:27.380 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:27.380+0000 7f9d62a1c640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T09:28:27.390 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:27.390+0000 7f9d5a028640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:28:27.414 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:27.413+0000 7f6615bed640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T09:28:27.424 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:27.424+0000 7f660d1f9640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:28:27.529 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-11T09:28:27.529 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":11,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:28:26.407355+0000","last_up_change":"2024-09-11T09:28:26.407355+0000","last_in_change":"2024-09-11T09:28:19.590242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6801","nonce":4208334809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6803","nonce":4208334809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6807","nonce":4208334809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6805","nonce":4208334809}]},"public_addr":"172.21.15.17:6801/4208334809","cluster_addr":"172.21.15.17:6803/4208334809","heartbeat_back_addr":"172.21.15.17:6807/4208334809","heartbeat_front_addr":"172.21.15.17:6805/4208334809","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:28:27.541 DEBUG:teuthology.misc:1 of 6 OSDs are up 2024-09-11T09:28:34.147 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T09:28:34.444 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-11T09:28:34.445 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":15,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:28:31.433297+0000","last_up_change":"2024-09-11T09:28:28.314478+0000","last_in_change":"2024-09-11T09:28:19.590242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T09:28:28.765227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6801","nonce":4208334809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6803","nonce":4208334809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6807","nonce":4208334809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6805","nonce":4208334809}]},"public_addr":"172.21.15.17:6801/4208334809","cluster_addr":"172.21.15.17:6803/4208334809","heartbeat_back_addr":"172.21.15.17:6807/4208334809","heartbeat_front_addr":"172.21.15.17:6805/4208334809","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6817","nonce":361445038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6819","nonce":361445038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6823","nonce":361445038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6821","nonce":361445038}]},"public_addr":"172.21.15.17:6817/361445038","cluster_addr":"172.21.15.17:6819/361445038","heartbeat_back_addr":"172.21.15.17:6823/361445038","heartbeat_front_addr":"172.21.15.17:6821/361445038","state":["exists","up"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6809","nonce":211521567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6811","nonce":211521567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6815","nonce":211521567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6813","nonce":211521567}]},"public_addr":"172.21.15.17:6809/211521567","cluster_addr":"172.21.15.17:6811/211521567","heartbeat_back_addr":"172.21.15.17:6815/211521567","heartbeat_front_addr":"172.21.15.17:6813/211521567","state":["exists","up"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6801","nonce":2070567113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6803","nonce":2070567113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6807","nonce":2070567113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6805","nonce":2070567113}]},"public_addr":"172.21.15.96:6801/2070567113","cluster_addr":"172.21.15.96:6803/2070567113","heartbeat_back_addr":"172.21.15.96:6807/2070567113","heartbeat_front_addr":"172.21.15.96:6805/2070567113","state":["exists","up"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6817","nonce":1795828331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6819","nonce":1795828331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6823","nonce":1795828331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6821","nonce":1795828331}]},"public_addr":"172.21.15.96:6817/1795828331","cluster_addr":"172.21.15.96:6819/1795828331","heartbeat_back_addr":"172.21.15.96:6823/1795828331","heartbeat_front_addr":"172.21.15.96:6821/1795828331","state":["exists","up"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6809","nonce":3808455039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6811","nonce":3808455039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6815","nonce":3808455039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6813","nonce":3808455039}]},"public_addr":"172.21.15.96:6809/3808455039","cluster_addr":"172.21.15.96:6811/3808455039","heartbeat_back_addr":"172.21.15.96:6815/3808455039","heartbeat_front_addr":"172.21.15.96:6813/3808455039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.320025+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.922939+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.876780+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.854159+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.081538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.168719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:28:34.456 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-11T09:28:34.457 INFO:tasks.ceph:Creating RBD pool 2024-09-11T09:28:34.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-11T09:28:35.446 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'rbd' created 2024-09-11T09:28:35.464 DEBUG:teuthology.orchestra.run.smithi017:> rbd --cluster ceph pool init rbd 2024-09-11T09:28:38.471 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T09:28:38.472 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T09:28:38.472 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T09:28:38.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T09:28:38.473 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T09:28:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:38.839 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":663661556},{"type":"v1","addr":"172.21.15.96:6825","nonce":663661556}]},"active_addr":"172.21.15.96:6825/663661556","active_change":"2024-09-11T09:28:24.715042+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3089360061}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":234940097}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2837122127}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2224320844}]}]} 2024-09-11T09:28:38.841 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T09:28:38.841 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T09:28:38.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T09:28:39.129 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:39.129 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":19,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:28:38.457841+0000","last_up_change":"2024-09-11T09:28:28.314478+0000","last_in_change":"2024-09-11T09:28:19.590242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T09:28:28.765227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T09:28:34.774979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6801","nonce":4208334809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6803","nonce":4208334809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6807","nonce":4208334809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6805","nonce":4208334809}]},"public_addr":"172.21.15.17:6801/4208334809","cluster_addr":"172.21.15.17:6803/4208334809","heartbeat_back_addr":"172.21.15.17:6807/4208334809","heartbeat_front_addr":"172.21.15.17:6805/4208334809","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6817","nonce":361445038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6819","nonce":361445038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6823","nonce":361445038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6821","nonce":361445038}]},"public_addr":"172.21.15.17:6817/361445038","cluster_addr":"172.21.15.17:6819/361445038","heartbeat_back_addr":"172.21.15.17:6823/361445038","heartbeat_front_addr":"172.21.15.17:6821/361445038","state":["exists","up"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6809","nonce":211521567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6811","nonce":211521567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6815","nonce":211521567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6813","nonce":211521567}]},"public_addr":"172.21.15.17:6809/211521567","cluster_addr":"172.21.15.17:6811/211521567","heartbeat_back_addr":"172.21.15.17:6815/211521567","heartbeat_front_addr":"172.21.15.17:6813/211521567","state":["exists","up"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6801","nonce":2070567113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6803","nonce":2070567113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6807","nonce":2070567113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6805","nonce":2070567113}]},"public_addr":"172.21.15.96:6801/2070567113","cluster_addr":"172.21.15.96:6803/2070567113","heartbeat_back_addr":"172.21.15.96:6807/2070567113","heartbeat_front_addr":"172.21.15.96:6805/2070567113","state":["exists","up"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6817","nonce":1795828331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6819","nonce":1795828331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6823","nonce":1795828331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6821","nonce":1795828331}]},"public_addr":"172.21.15.96:6817/1795828331","cluster_addr":"172.21.15.96:6819/1795828331","heartbeat_back_addr":"172.21.15.96:6823/1795828331","heartbeat_front_addr":"172.21.15.96:6821/1795828331","state":["exists","up"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6809","nonce":3808455039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6811","nonce":3808455039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6815","nonce":3808455039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6813","nonce":3808455039}]},"public_addr":"172.21.15.96:6809/3808455039","cluster_addr":"172.21.15.96:6811/3808455039","heartbeat_back_addr":"172.21.15.96:6815/3808455039","heartbeat_front_addr":"172.21.15.96:6813/3808455039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.320025+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.922939+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.876780+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.854159+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.081538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.168719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:28:39.142 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T09:28:39.142 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T09:28:39.432 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:39.433 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":19,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:28:38.457841+0000","last_up_change":"2024-09-11T09:28:28.314478+0000","last_in_change":"2024-09-11T09:28:19.590242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T09:28:28.765227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T09:28:34.774979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6801","nonce":4208334809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6803","nonce":4208334809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6807","nonce":4208334809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":4208334809},{"type":"v1","addr":"172.21.15.17:6805","nonce":4208334809}]},"public_addr":"172.21.15.17:6801/4208334809","cluster_addr":"172.21.15.17:6803/4208334809","heartbeat_back_addr":"172.21.15.17:6807/4208334809","heartbeat_front_addr":"172.21.15.17:6805/4208334809","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6817","nonce":361445038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6819","nonce":361445038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6823","nonce":361445038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6821","nonce":361445038}]},"public_addr":"172.21.15.17:6817/361445038","cluster_addr":"172.21.15.17:6819/361445038","heartbeat_back_addr":"172.21.15.17:6823/361445038","heartbeat_front_addr":"172.21.15.17:6821/361445038","state":["exists","up"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6809","nonce":211521567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6811","nonce":211521567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6815","nonce":211521567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6813","nonce":211521567}]},"public_addr":"172.21.15.17:6809/211521567","cluster_addr":"172.21.15.17:6811/211521567","heartbeat_back_addr":"172.21.15.17:6815/211521567","heartbeat_front_addr":"172.21.15.17:6813/211521567","state":["exists","up"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6801","nonce":2070567113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6803","nonce":2070567113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6807","nonce":2070567113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6805","nonce":2070567113}]},"public_addr":"172.21.15.96:6801/2070567113","cluster_addr":"172.21.15.96:6803/2070567113","heartbeat_back_addr":"172.21.15.96:6807/2070567113","heartbeat_front_addr":"172.21.15.96:6805/2070567113","state":["exists","up"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6817","nonce":1795828331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6819","nonce":1795828331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6823","nonce":1795828331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6821","nonce":1795828331}]},"public_addr":"172.21.15.96:6817/1795828331","cluster_addr":"172.21.15.96:6819/1795828331","heartbeat_back_addr":"172.21.15.96:6823/1795828331","heartbeat_front_addr":"172.21.15.96:6821/1795828331","state":["exists","up"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6809","nonce":3808455039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6811","nonce":3808455039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6815","nonce":3808455039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6813","nonce":3808455039}]},"public_addr":"172.21.15.96:6809/3808455039","cluster_addr":"172.21.15.96:6811/3808455039","heartbeat_back_addr":"172.21.15.96:6815/3808455039","heartbeat_front_addr":"172.21.15.96:6813/3808455039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.320025+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.922939+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.876780+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.854159+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.081538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.168719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:28:39.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T09:28:39.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T09:28:39.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T09:28:39.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T09:28:39.447 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T09:28:39.448 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T09:28:39.605 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:39.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T09:28:39.606 INFO:teuthology.orchestra.run.smithi017.stdout:47244640260 2024-09-11T09:28:39.607 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T09:28:39.652 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:39.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T09:28:39.655 INFO:teuthology.orchestra.run.smithi017.stdout:51539607558 2024-09-11T09:28:39.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T09:28:39.657 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:39.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T09:28:39.658 INFO:teuthology.orchestra.run.smithi017.stdout:51539607558 2024-09-11T09:28:39.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T09:28:40.054 INFO:teuthology.orchestra.run.smithi017.stdout:51539607556 2024-09-11T09:28:40.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.3 2024-09-11T09:28:40.070 INFO:teuthology.orchestra.run.smithi017.stdout:51539607556 2024-09-11T09:28:40.083 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.4 2024-09-11T09:28:40.130 INFO:teuthology.orchestra.run.smithi017.stdout:47244640259 2024-09-11T09:28:40.142 INFO:teuthology.orchestra.run.smithi017.stdout:51539607556 2024-09-11T09:28:40.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640259 for osd.0 2024-09-11T09:28:40.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.5 2024-09-11T09:28:40.227 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:40.227 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:40.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.2 2024-09-11T09:28:40.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607557 for osd.1 2024-09-11T09:28:41.069 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T09:28:41.084 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T09:28:41.145 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T09:28:41.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T09:28:41.246 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T09:28:41.246 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T09:28:41.405 INFO:teuthology.orchestra.run.smithi017.stdout:51539607556 2024-09-11T09:28:41.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.3 2024-09-11T09:28:41.463 INFO:teuthology.orchestra.run.smithi017.stdout:51539607556 2024-09-11T09:28:41.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.4 2024-09-11T09:28:41.602 INFO:teuthology.orchestra.run.smithi017.stdout:51539607558 2024-09-11T09:28:41.612 INFO:teuthology.orchestra.run.smithi017.stdout:51539607558 2024-09-11T09:28:41.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.1 2024-09-11T09:28:41.615 DEBUG:teuthology.parallel:result is None 2024-09-11T09:28:41.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607558 got 51539607558 for osd.2 2024-09-11T09:28:41.626 DEBUG:teuthology.parallel:result is None 2024-09-11T09:28:41.659 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:41.668 INFO:teuthology.orchestra.run.smithi017.stdout:47244640260 2024-09-11T09:28:41.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.5 2024-09-11T09:28:41.674 DEBUG:teuthology.parallel:result is None 2024-09-11T09:28:41.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.0 2024-09-11T09:28:41.682 DEBUG:teuthology.parallel:result is None 2024-09-11T09:28:42.427 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T09:28:42.479 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T09:28:42.741 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:42.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.3 2024-09-11T09:28:42.754 DEBUG:teuthology.parallel:result is None 2024-09-11T09:28:42.785 INFO:teuthology.orchestra.run.smithi017.stdout:51539607557 2024-09-11T09:28:42.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.4 2024-09-11T09:28:42.798 DEBUG:teuthology.parallel:result is None 2024-09-11T09:28:42.798 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T09:28:42.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:28:43.126 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:43.126 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:28:43.138 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-11T09:28:42.723274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":175856,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566055184,"statfs":{"total":579820584960,"available":579640508416,"internally_reserved":0,"allocated":1204224,"data_stored":1537642,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001287"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615295+0000","last_change":"2024-09-11T09:28:38.615508+0000","last_active":"2024-09-11T09:28:38.615295+0000","last_peered":"2024-09-11T09:28:38.615295+0000","last_clean":"2024-09-11T09:28:38.615295+0000","last_became_active":"2024-09-11T09:28:36.535007+0000","last_became_peered":"2024-09-11T09:28:36.535007+0000","last_unstale":"2024-09-11T09:28:38.615295+0000","last_undegraded":"2024-09-11T09:28:38.615295+0000","last_fullsized":"2024-09-11T09:28:38.615295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:46.010169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068309000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461787+0000","last_change":"2024-09-11T09:28:38.462223+0000","last_active":"2024-09-11T09:28:38.461787+0000","last_peered":"2024-09-11T09:28:38.461787+0000","last_clean":"2024-09-11T09:28:38.461787+0000","last_became_active":"2024-09-11T09:28:36.455841+0000","last_became_peered":"2024-09-11T09:28:36.455841+0000","last_unstale":"2024-09-11T09:28:38.461787+0000","last_undegraded":"2024-09-11T09:28:38.461787+0000","last_fullsized":"2024-09-11T09:28:38.461787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:26.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615011+0000","last_change":"2024-09-11T09:28:38.615243+0000","last_active":"2024-09-11T09:28:38.615011+0000","last_peered":"2024-09-11T09:28:38.615011+0000","last_clean":"2024-09-11T09:28:38.615011+0000","last_became_active":"2024-09-11T09:28:36.453319+0000","last_became_peered":"2024-09-11T09:28:36.453319+0000","last_unstale":"2024-09-11T09:28:38.615011+0000","last_undegraded":"2024-09-11T09:28:38.615011+0000","last_fullsized":"2024-09-11T09:28:38.615011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:41.513882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047127699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461961+0000","last_change":"2024-09-11T09:28:38.462264+0000","last_active":"2024-09-11T09:28:38.461961+0000","last_peered":"2024-09-11T09:28:38.461961+0000","last_clean":"2024-09-11T09:28:38.461961+0000","last_became_active":"2024-09-11T09:28:36.454444+0000","last_became_peered":"2024-09-11T09:28:36.454444+0000","last_unstale":"2024-09-11T09:28:38.461961+0000","last_undegraded":"2024-09-11T09:28:38.461961+0000","last_fullsized":"2024-09-11T09:28:38.461961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:56.449791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050275600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.467426+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:28:38.467426+0000","last_peered":"2024-09-11T09:28:38.467426+0000","last_clean":"2024-09-11T09:28:38.467426+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:28:38.467426+0000","last_undegraded":"2024-09-11T09:28:38.467426+0000","last_fullsized":"2024-09-11T09:28:38.467426+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461410+0000","last_change":"2024-09-11T09:28:38.461756+0000","last_active":"2024-09-11T09:28:38.461410+0000","last_peered":"2024-09-11T09:28:38.461410+0000","last_clean":"2024-09-11T09:28:38.461410+0000","last_became_active":"2024-09-11T09:28:36.456803+0000","last_became_peered":"2024-09-11T09:28:36.456803+0000","last_unstale":"2024-09-11T09:28:38.461410+0000","last_undegraded":"2024-09-11T09:28:38.461410+0000","last_fullsized":"2024-09-11T09:28:38.461410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:51:50.314877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057627399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614964+0000","last_change":"2024-09-11T09:28:38.615249+0000","last_active":"2024-09-11T09:28:38.614964+0000","last_peered":"2024-09-11T09:28:38.614964+0000","last_clean":"2024-09-11T09:28:38.614964+0000","last_became_active":"2024-09-11T09:28:36.453844+0000","last_became_peered":"2024-09-11T09:28:36.453844+0000","last_unstale":"2024-09-11T09:28:38.614964+0000","last_undegraded":"2024-09-11T09:28:38.614964+0000","last_fullsized":"2024-09-11T09:28:38.614964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:37.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047575299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.466601+0000","last_change":"2024-09-11T09:28:38.466834+0000","last_active":"2024-09-11T09:28:38.466601+0000","last_peered":"2024-09-11T09:28:38.466601+0000","last_clean":"2024-09-11T09:28:38.466601+0000","last_became_active":"2024-09-11T09:28:36.457672+0000","last_became_peered":"2024-09-11T09:28:36.457672+0000","last_unstale":"2024-09-11T09:28:38.466601+0000","last_undegraded":"2024-09-11T09:28:38.466601+0000","last_fullsized":"2024-09-11T09:28:38.466601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:12.525689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614710+0000","last_change":"2024-09-11T09:28:30.806126+0000","last_active":"2024-09-11T09:28:38.614710+0000","last_peered":"2024-09-11T09:28:38.614710+0000","last_clean":"2024-09-11T09:28:38.614710+0000","last_became_active":"2024-09-11T09:28:30.805162+0000","last_became_peered":"2024-09-11T09:28:30.805162+0000","last_unstale":"2024-09-11T09:28:38.614710+0000","last_undegraded":"2024-09-11T09:28:38.614710+0000","last_fullsized":"2024-09-11T09:28:38.614710+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:57.740921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:28:43.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:28:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:43.426 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:28:43.437 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-11T09:28:42.723274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":175856,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566055184,"statfs":{"total":579820584960,"available":579640508416,"internally_reserved":0,"allocated":1204224,"data_stored":1537642,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001287"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615295+0000","last_change":"2024-09-11T09:28:38.615508+0000","last_active":"2024-09-11T09:28:38.615295+0000","last_peered":"2024-09-11T09:28:38.615295+0000","last_clean":"2024-09-11T09:28:38.615295+0000","last_became_active":"2024-09-11T09:28:36.535007+0000","last_became_peered":"2024-09-11T09:28:36.535007+0000","last_unstale":"2024-09-11T09:28:38.615295+0000","last_undegraded":"2024-09-11T09:28:38.615295+0000","last_fullsized":"2024-09-11T09:28:38.615295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:46.010169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068309000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461787+0000","last_change":"2024-09-11T09:28:38.462223+0000","last_active":"2024-09-11T09:28:38.461787+0000","last_peered":"2024-09-11T09:28:38.461787+0000","last_clean":"2024-09-11T09:28:38.461787+0000","last_became_active":"2024-09-11T09:28:36.455841+0000","last_became_peered":"2024-09-11T09:28:36.455841+0000","last_unstale":"2024-09-11T09:28:38.461787+0000","last_undegraded":"2024-09-11T09:28:38.461787+0000","last_fullsized":"2024-09-11T09:28:38.461787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:26.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615011+0000","last_change":"2024-09-11T09:28:38.615243+0000","last_active":"2024-09-11T09:28:38.615011+0000","last_peered":"2024-09-11T09:28:38.615011+0000","last_clean":"2024-09-11T09:28:38.615011+0000","last_became_active":"2024-09-11T09:28:36.453319+0000","last_became_peered":"2024-09-11T09:28:36.453319+0000","last_unstale":"2024-09-11T09:28:38.615011+0000","last_undegraded":"2024-09-11T09:28:38.615011+0000","last_fullsized":"2024-09-11T09:28:38.615011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:41.513882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047127699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461961+0000","last_change":"2024-09-11T09:28:38.462264+0000","last_active":"2024-09-11T09:28:38.461961+0000","last_peered":"2024-09-11T09:28:38.461961+0000","last_clean":"2024-09-11T09:28:38.461961+0000","last_became_active":"2024-09-11T09:28:36.454444+0000","last_became_peered":"2024-09-11T09:28:36.454444+0000","last_unstale":"2024-09-11T09:28:38.461961+0000","last_undegraded":"2024-09-11T09:28:38.461961+0000","last_fullsized":"2024-09-11T09:28:38.461961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:56.449791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050275600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.467426+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:28:38.467426+0000","last_peered":"2024-09-11T09:28:38.467426+0000","last_clean":"2024-09-11T09:28:38.467426+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:28:38.467426+0000","last_undegraded":"2024-09-11T09:28:38.467426+0000","last_fullsized":"2024-09-11T09:28:38.467426+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461410+0000","last_change":"2024-09-11T09:28:38.461756+0000","last_active":"2024-09-11T09:28:38.461410+0000","last_peered":"2024-09-11T09:28:38.461410+0000","last_clean":"2024-09-11T09:28:38.461410+0000","last_became_active":"2024-09-11T09:28:36.456803+0000","last_became_peered":"2024-09-11T09:28:36.456803+0000","last_unstale":"2024-09-11T09:28:38.461410+0000","last_undegraded":"2024-09-11T09:28:38.461410+0000","last_fullsized":"2024-09-11T09:28:38.461410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:51:50.314877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057627399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614964+0000","last_change":"2024-09-11T09:28:38.615249+0000","last_active":"2024-09-11T09:28:38.614964+0000","last_peered":"2024-09-11T09:28:38.614964+0000","last_clean":"2024-09-11T09:28:38.614964+0000","last_became_active":"2024-09-11T09:28:36.453844+0000","last_became_peered":"2024-09-11T09:28:36.453844+0000","last_unstale":"2024-09-11T09:28:38.614964+0000","last_undegraded":"2024-09-11T09:28:38.614964+0000","last_fullsized":"2024-09-11T09:28:38.614964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:37.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047575299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.466601+0000","last_change":"2024-09-11T09:28:38.466834+0000","last_active":"2024-09-11T09:28:38.466601+0000","last_peered":"2024-09-11T09:28:38.466601+0000","last_clean":"2024-09-11T09:28:38.466601+0000","last_became_active":"2024-09-11T09:28:36.457672+0000","last_became_peered":"2024-09-11T09:28:36.457672+0000","last_unstale":"2024-09-11T09:28:38.466601+0000","last_undegraded":"2024-09-11T09:28:38.466601+0000","last_fullsized":"2024-09-11T09:28:38.466601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:12.525689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614710+0000","last_change":"2024-09-11T09:28:30.806126+0000","last_active":"2024-09-11T09:28:38.614710+0000","last_peered":"2024-09-11T09:28:38.614710+0000","last_clean":"2024-09-11T09:28:38.614710+0000","last_became_active":"2024-09-11T09:28:30.805162+0000","last_became_peered":"2024-09-11T09:28:30.805162+0000","last_unstale":"2024-09-11T09:28:38.614710+0000","last_undegraded":"2024-09-11T09:28:38.614710+0000","last_fullsized":"2024-09-11T09:28:38.614710+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:57.740921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:28:43.438 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T09:28:43.438 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T09:28:43.438 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T09:28:43.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T09:28:43.763 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:43.763 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T09:28:43.775 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T09:28:43.775 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-11T09:28:43.788 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-11T09:28:43.788 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'daaf78de334647fc0c800f4ac495cc9475a26bae'}} 2024-09-11T09:28:43.788 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-11T09:28:43.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-11T09:28:44.099 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-11T09:28:44.099 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T09:28:44.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:28:44.383 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:28:44.394 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-11T09:28:42.723274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":175856,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566055184,"statfs":{"total":579820584960,"available":579640508416,"internally_reserved":0,"allocated":1204224,"data_stored":1537642,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001287"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615295+0000","last_change":"2024-09-11T09:28:38.615508+0000","last_active":"2024-09-11T09:28:38.615295+0000","last_peered":"2024-09-11T09:28:38.615295+0000","last_clean":"2024-09-11T09:28:38.615295+0000","last_became_active":"2024-09-11T09:28:36.535007+0000","last_became_peered":"2024-09-11T09:28:36.535007+0000","last_unstale":"2024-09-11T09:28:38.615295+0000","last_undegraded":"2024-09-11T09:28:38.615295+0000","last_fullsized":"2024-09-11T09:28:38.615295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:46.010169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068309000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461787+0000","last_change":"2024-09-11T09:28:38.462223+0000","last_active":"2024-09-11T09:28:38.461787+0000","last_peered":"2024-09-11T09:28:38.461787+0000","last_clean":"2024-09-11T09:28:38.461787+0000","last_became_active":"2024-09-11T09:28:36.455841+0000","last_became_peered":"2024-09-11T09:28:36.455841+0000","last_unstale":"2024-09-11T09:28:38.461787+0000","last_undegraded":"2024-09-11T09:28:38.461787+0000","last_fullsized":"2024-09-11T09:28:38.461787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:26.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615011+0000","last_change":"2024-09-11T09:28:38.615243+0000","last_active":"2024-09-11T09:28:38.615011+0000","last_peered":"2024-09-11T09:28:38.615011+0000","last_clean":"2024-09-11T09:28:38.615011+0000","last_became_active":"2024-09-11T09:28:36.453319+0000","last_became_peered":"2024-09-11T09:28:36.453319+0000","last_unstale":"2024-09-11T09:28:38.615011+0000","last_undegraded":"2024-09-11T09:28:38.615011+0000","last_fullsized":"2024-09-11T09:28:38.615011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:41.513882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047127699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461961+0000","last_change":"2024-09-11T09:28:38.462264+0000","last_active":"2024-09-11T09:28:38.461961+0000","last_peered":"2024-09-11T09:28:38.461961+0000","last_clean":"2024-09-11T09:28:38.461961+0000","last_became_active":"2024-09-11T09:28:36.454444+0000","last_became_peered":"2024-09-11T09:28:36.454444+0000","last_unstale":"2024-09-11T09:28:38.461961+0000","last_undegraded":"2024-09-11T09:28:38.461961+0000","last_fullsized":"2024-09-11T09:28:38.461961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:56.449791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050275600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.467426+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:28:38.467426+0000","last_peered":"2024-09-11T09:28:38.467426+0000","last_clean":"2024-09-11T09:28:38.467426+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:28:38.467426+0000","last_undegraded":"2024-09-11T09:28:38.467426+0000","last_fullsized":"2024-09-11T09:28:38.467426+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461410+0000","last_change":"2024-09-11T09:28:38.461756+0000","last_active":"2024-09-11T09:28:38.461410+0000","last_peered":"2024-09-11T09:28:38.461410+0000","last_clean":"2024-09-11T09:28:38.461410+0000","last_became_active":"2024-09-11T09:28:36.456803+0000","last_became_peered":"2024-09-11T09:28:36.456803+0000","last_unstale":"2024-09-11T09:28:38.461410+0000","last_undegraded":"2024-09-11T09:28:38.461410+0000","last_fullsized":"2024-09-11T09:28:38.461410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:51:50.314877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057627399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614964+0000","last_change":"2024-09-11T09:28:38.615249+0000","last_active":"2024-09-11T09:28:38.614964+0000","last_peered":"2024-09-11T09:28:38.614964+0000","last_clean":"2024-09-11T09:28:38.614964+0000","last_became_active":"2024-09-11T09:28:36.453844+0000","last_became_peered":"2024-09-11T09:28:36.453844+0000","last_unstale":"2024-09-11T09:28:38.614964+0000","last_undegraded":"2024-09-11T09:28:38.614964+0000","last_fullsized":"2024-09-11T09:28:38.614964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:37.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047575299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.466601+0000","last_change":"2024-09-11T09:28:38.466834+0000","last_active":"2024-09-11T09:28:38.466601+0000","last_peered":"2024-09-11T09:28:38.466601+0000","last_clean":"2024-09-11T09:28:38.466601+0000","last_became_active":"2024-09-11T09:28:36.457672+0000","last_became_peered":"2024-09-11T09:28:36.457672+0000","last_unstale":"2024-09-11T09:28:38.466601+0000","last_undegraded":"2024-09-11T09:28:38.466601+0000","last_fullsized":"2024-09-11T09:28:38.466601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:12.525689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614710+0000","last_change":"2024-09-11T09:28:30.806126+0000","last_active":"2024-09-11T09:28:38.614710+0000","last_peered":"2024-09-11T09:28:38.614710+0000","last_clean":"2024-09-11T09:28:38.614710+0000","last_became_active":"2024-09-11T09:28:30.805162+0000","last_became_peered":"2024-09-11T09:28:30.805162+0000","last_unstale":"2024-09-11T09:28:38.614710+0000","last_undegraded":"2024-09-11T09:28:38.614710+0000","last_fullsized":"2024-09-11T09:28:38.614710+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:57.740921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:28:44.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:28:44.679 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:28:44.680 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:28:44.692 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-11T09:28:42.723274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":175856,"kb_used_data":1176,"kb_used_omap":9,"kb_used_meta":174518,"kb_avail":566055184,"statfs":{"total":579820584960,"available":579640508416,"internally_reserved":0,"allocated":1204224,"data_stored":1537642,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":9568,"internal_metadata":178707104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001287"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615295+0000","last_change":"2024-09-11T09:28:38.615508+0000","last_active":"2024-09-11T09:28:38.615295+0000","last_peered":"2024-09-11T09:28:38.615295+0000","last_clean":"2024-09-11T09:28:38.615295+0000","last_became_active":"2024-09-11T09:28:36.535007+0000","last_became_peered":"2024-09-11T09:28:36.535007+0000","last_unstale":"2024-09-11T09:28:38.615295+0000","last_undegraded":"2024-09-11T09:28:38.615295+0000","last_fullsized":"2024-09-11T09:28:38.615295+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:46.010169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068309000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461787+0000","last_change":"2024-09-11T09:28:38.462223+0000","last_active":"2024-09-11T09:28:38.461787+0000","last_peered":"2024-09-11T09:28:38.461787+0000","last_clean":"2024-09-11T09:28:38.461787+0000","last_became_active":"2024-09-11T09:28:36.455841+0000","last_became_peered":"2024-09-11T09:28:36.455841+0000","last_unstale":"2024-09-11T09:28:38.461787+0000","last_undegraded":"2024-09-11T09:28:38.461787+0000","last_fullsized":"2024-09-11T09:28:38.461787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:26.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.615011+0000","last_change":"2024-09-11T09:28:38.615243+0000","last_active":"2024-09-11T09:28:38.615011+0000","last_peered":"2024-09-11T09:28:38.615011+0000","last_clean":"2024-09-11T09:28:38.615011+0000","last_became_active":"2024-09-11T09:28:36.453319+0000","last_became_peered":"2024-09-11T09:28:36.453319+0000","last_unstale":"2024-09-11T09:28:38.615011+0000","last_undegraded":"2024-09-11T09:28:38.615011+0000","last_fullsized":"2024-09-11T09:28:38.615011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:41.513882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047127699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461961+0000","last_change":"2024-09-11T09:28:38.462264+0000","last_active":"2024-09-11T09:28:38.461961+0000","last_peered":"2024-09-11T09:28:38.461961+0000","last_clean":"2024-09-11T09:28:38.461961+0000","last_became_active":"2024-09-11T09:28:36.454444+0000","last_became_peered":"2024-09-11T09:28:36.454444+0000","last_unstale":"2024-09-11T09:28:38.461961+0000","last_undegraded":"2024-09-11T09:28:38.461961+0000","last_fullsized":"2024-09-11T09:28:38.461961+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:56.449791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050275600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.467426+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:28:38.467426+0000","last_peered":"2024-09-11T09:28:38.467426+0000","last_clean":"2024-09-11T09:28:38.467426+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:28:38.467426+0000","last_undegraded":"2024-09-11T09:28:38.467426+0000","last_fullsized":"2024-09-11T09:28:38.467426+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.461410+0000","last_change":"2024-09-11T09:28:38.461756+0000","last_active":"2024-09-11T09:28:38.461410+0000","last_peered":"2024-09-11T09:28:38.461410+0000","last_clean":"2024-09-11T09:28:38.461410+0000","last_became_active":"2024-09-11T09:28:36.456803+0000","last_became_peered":"2024-09-11T09:28:36.456803+0000","last_unstale":"2024-09-11T09:28:38.461410+0000","last_undegraded":"2024-09-11T09:28:38.461410+0000","last_fullsized":"2024-09-11T09:28:38.461410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:51:50.314877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057627399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614964+0000","last_change":"2024-09-11T09:28:38.615249+0000","last_active":"2024-09-11T09:28:38.614964+0000","last_peered":"2024-09-11T09:28:38.614964+0000","last_clean":"2024-09-11T09:28:38.614964+0000","last_became_active":"2024-09-11T09:28:36.453844+0000","last_became_peered":"2024-09-11T09:28:36.453844+0000","last_unstale":"2024-09-11T09:28:38.614964+0000","last_undegraded":"2024-09-11T09:28:38.614964+0000","last_fullsized":"2024-09-11T09:28:38.614964+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:37.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047575299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.466601+0000","last_change":"2024-09-11T09:28:38.466834+0000","last_active":"2024-09-11T09:28:38.466601+0000","last_peered":"2024-09-11T09:28:38.466601+0000","last_clean":"2024-09-11T09:28:38.466601+0000","last_became_active":"2024-09-11T09:28:36.457672+0000","last_became_peered":"2024-09-11T09:28:36.457672+0000","last_unstale":"2024-09-11T09:28:38.466601+0000","last_undegraded":"2024-09-11T09:28:38.466601+0000","last_fullsized":"2024-09-11T09:28:38.466601+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:05:12.525689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-11T09:28:38.614710+0000","last_change":"2024-09-11T09:28:30.806126+0000","last_active":"2024-09-11T09:28:38.614710+0000","last_peered":"2024-09-11T09:28:38.614710+0000","last_clean":"2024-09-11T09:28:38.614710+0000","last_became_active":"2024-09-11T09:28:30.805162+0000","last_became_peered":"2024-09-11T09:28:30.805162+0000","last_unstale":"2024-09-11T09:28:38.614710+0000","last_undegraded":"2024-09-11T09:28:38.614710+0000","last_fullsized":"2024-09-11T09:28:38.614710+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:57.740921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607558,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":649846,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:28:44.693 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T09:28:44.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 20 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:fsid 23a8bf95-caba-4fea-a762-61040a1c1b77 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-09-11T09:28:17.894525+0000 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:modified 2024-09-11T09:28:42.675327+0000 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:crush_version 5 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:full_ratio 0.95 2024-09-11T09:28:44.984 INFO:teuthology.orchestra.run.smithi017.stdout:backfillfull_ratio 0.9 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:nearfull_ratio 0.85 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:require_min_compat_client luminous 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:min_compat_client jewel 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:require_osd_release squid 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:stretch_mode_enabled false 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:max_osd 6 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6800/4208334809,v1:172.21.15.17:6801/4208334809] [v2:172.21.15.17:6802/4208334809,v1:172.21.15.17:6803/4208334809] exists,up 44ce15f5-731e-4d58-9dd7-bd8f186e5783 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6816/361445038,v1:172.21.15.17:6817/361445038] [v2:172.21.15.17:6818/361445038,v1:172.21.15.17:6819/361445038] exists,up dba82e38-a74b-4d42-bad4-9b1e013d4da3 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6808/211521567,v1:172.21.15.17:6809/211521567] [v2:172.21.15.17:6810/211521567,v1:172.21.15.17:6811/211521567] exists,up 64a33918-9bb2-4d59-a66e-73da15d78cbd 2024-09-11T09:28:44.985 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/2070567113,v1:172.21.15.96:6801/2070567113] [v2:172.21.15.96:6802/2070567113,v1:172.21.15.96:6803/2070567113] exists,up a4774bc9-54ea-4272-b6b4-a6dd00c4c930 2024-09-11T09:28:44.986 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/1795828331,v1:172.21.15.96:6817/1795828331] [v2:172.21.15.96:6818/1795828331,v1:172.21.15.96:6819/1795828331] exists,up a3984c6d-7250-4e85-a032-1dc0315642d8 2024-09-11T09:28:44.986 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/3808455039,v1:172.21.15.96:6809/3808455039] [v2:172.21.15.96:6810/3808455039,v1:172.21.15.96:6811/3808455039] exists,up 0a40bbb7-9d37-4a1b-b525-86608a3eaaa2 2024-09-11T09:28:44.996 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6800/4208334809,v1:172.21.15.17:6801/4208334809] [v2:172.21.15.17:6802/4208334809,v1:172.21.15.17:6803/4208334809] exists,up 44ce15f5-731e-4d58-9dd7-bd8f186e5783', 'osd.1 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6816/361445038,v1:172.21.15.17:6817/361445038] [v2:172.21.15.17:6818/361445038,v1:172.21.15.17:6819/361445038] exists,up dba82e38-a74b-4d42-bad4-9b1e013d4da3', 'osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6808/211521567,v1:172.21.15.17:6809/211521567] [v2:172.21.15.17:6810/211521567,v1:172.21.15.17:6811/211521567] exists,up 64a33918-9bb2-4d59-a66e-73da15d78cbd', 'osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/2070567113,v1:172.21.15.96:6801/2070567113] [v2:172.21.15.96:6802/2070567113,v1:172.21.15.96:6803/2070567113] exists,up a4774bc9-54ea-4272-b6b4-a6dd00c4c930', 'osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/1795828331,v1:172.21.15.96:6817/1795828331] [v2:172.21.15.96:6818/1795828331,v1:172.21.15.96:6819/1795828331] exists,up a3984c6d-7250-4e85-a032-1dc0315642d8', 'osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/3808455039,v1:172.21.15.96:6809/3808455039] [v2:172.21.15.96:6810/3808455039,v1:172.21.15.96:6811/3808455039] exists,up 0a40bbb7-9d37-4a1b-b525-86608a3eaaa2'] 2024-09-11T09:28:44.997 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "mon.a", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster": "ceph", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "admin_socket_mode": "", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_client_required": "cephx, none", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_cluster_required": "cephx", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_debug": "true", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-11T09:28:45.099 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_service_required": "cephx", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_supported": "cephx", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio": "true", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio_poll_ms": "250", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_aio_reap_max": "16", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_async_discard": "false", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_block_size": "4096", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_aio": "false", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-11T09:28:45.100 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_debug_inflight_ios": "false", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_enable_discard": "false", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_flock_retry": "3", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_inject_crash": "0", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_ioring": "false", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_ioring_hipri": "false", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-11T09:28:45.101 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bdev_type": "", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_alloc_size": "1048576", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_allocator": "hybrid", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_buffered_io": "true", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_check_for_zeros": "false", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_compact_log_sync": "false", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-11T09:28:45.102 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_max_log_runway": "4194304", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_max_prefetch": "1048576", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_min_flush_size": "524288", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_min_log_runway": "1048576", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_replay_recovery": "false", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_sync_write": "false", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-11T09:28:45.103 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_allocation_from_file": "true", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_allocator": "hybrid", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bdev_label_multi": "true", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-11T09:28:45.104 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_create": "true", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_db_create": "false", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_db_path": "", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_db_size": "0", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_path": "", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_preallocate_file": "false", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_size": "107374182400", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_wal_create": "false", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_wal_path": "", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_block_wal_size": "100663296", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs": "true", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-11T09:28:45.105 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_autotune": "true", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_size": "0", 2024-09-11T09:28:45.106 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_type": "2q", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_clone_cow": "true", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-11T09:28:45.107 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_mode": "none", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_csum_type": "crc32c", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_freelist": "false", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-11T09:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_small_allocations": "0", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_default_buffered_read": "true", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_default_buffered_write": "false", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-11T09:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fail_eio": "false", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-11T09:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mount": "false", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_umount": "false", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-11T09:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_ignore_data_csum": "false", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_log_op_age": "5.000000", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_alloc_size": "0", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_blob_size": "0", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_max_deferred_txc": "32", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_min_alloc_size": "0", 2024-09-11T09:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_nid_prealloc": "1024", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_retry_disk_reads": "3", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_cf": "true", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-11T09:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_spdk_mem": "512", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_tracing": "false", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-11T09:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_zero_block_detection": "false", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-11T09:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_cluster_require_signatures": "false", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_cluster_require_version": "2", 2024-09-11T09:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_require_signatures": "false", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_require_version": "2", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_service_require_signatures": "false", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_service_require_version": "2", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "cephx_sign_messages": "true", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "chdir": "", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_acl_type": "", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_asio_thread_count": "2", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_cache_mid": "0.750000", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_cache_size": "16384", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_caps_release_delay": "5", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_check_pool_perm": "true", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-11T09:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_force_sync_read": "false", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_getattr_caps": "false", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_inject_features": "", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_debug_inject_tick_delay": "0", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_die_on_failed_remount": "false", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_dirsize_rbytes": "true", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_force_lazyio": "false", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_fs": "", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_inject_release_failure": "false", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_max_inline_size": "4096", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mds_namespace": "", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_metadata": "", 2024-09-11T09:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mount_gid": "-1", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mount_timeout": "300", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mount_uid": "-1", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_mountpoint": "/", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_notify_timeout": "10", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc": "true", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_max_dirty": "104857600", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_max_objects": "1000", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_size": "209715200", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_oc_target_dirty": "8388608", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_permissions": "true", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_quota": "true", 2024-09-11T09:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_quota_df": "true", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_readahead_max_bytes": "0", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_readahead_max_periods": "4", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_readahead_min": "131072", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_reconnect_stale": "false", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_shutdown_timeout": "30", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_snapdir": ".snap", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_tick_interval": "1", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_trace": "", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_try_dentry_invalidate": "false", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_use_faked_inos": "false", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_use_random_mds": "false", 2024-09-11T09:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_graylog": "false", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_graylog_port": "12201", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_monitors": "default=true", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_syslog": "false", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "clog_to_syslog_level": "info", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "-", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_network": "", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_network_interface": "", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zlib_isal": "false", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zlib_level": "5", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zlib_winsize": "-15", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "compressor_zstd_level": "1", 2024-09-11T09:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_alien_op_num_threads": "6", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_osd_stat_interval": "0", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_seastar_cpu_cores": "", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crimson_seastar_num_threads": "0", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location_hook": "", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location_hook_timeout": "10", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "daemonize": "false", 2024-09-11T09:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "daos_pool": "tank", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_alienstore": "0/5", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_allow_any_pool_priority": "false", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_asok": "1/5", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_asok_assert_abort": "false", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_asserts_on_shutdown": "true", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_auth": "20/20", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_bdev": "1/3", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_bluefs": "1/5", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_bluestore": "1/5", 2024-09-11T09:28:45.123 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_buffer": "0/1", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_ceph_exporter": "1/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_cephfs_mirror": "0/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_cephsqlite": "0/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_client": "0/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_compressor": "1/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_context": "0/1", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_crimson_interrupt": "0/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_crush": "1/1", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_crypto": "1/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_cyanstore": "0/5", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_deliberately_leak_memory": "false", 2024-09-11T09:28:45.124 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_disable_randomized_ping": "false", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_dpdk": "1/5", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_eventtrace": "1/5", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_filer": "0/1", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_filestore": "1/3", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_finisher": "1/1", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_fuse": "1/5", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_heartbeat_testing_span": "0", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_heartbeatmap": "1/5", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_javaclient": "1/5", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_journal": "1/3", 2024-09-11T09:28:45.125 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_journaler": "0/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_kstore": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_lockdep": "0/1", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mclock": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_balancer": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_locker": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_log": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_log_expire": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_migrator": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mds_quiesce": "3/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_memstore": "1/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mgr": "2/5", 2024-09-11T09:28:45.126 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mgrc": "1/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_mon": "20/20", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_monc": "0/10", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_ms": "1/1", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_none": "0/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_objclass": "0/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_objectcacher": "0/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_objecter": "0/1", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_optracker": "0/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_osd": "1/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_paxos": "20/20", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_perfcounter": "1/5", 2024-09-11T09:28:45.127 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_prioritycache": "1/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rados": "0/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd": "0/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd_mirror": "0/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd_pwl": "0/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rbd_replay": "0/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_refs": "0/0", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_reserver": "1/1", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw": "1/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_access": "1/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_datacache": "1/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_dbstore": "1/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_flight": "1/5", 2024-09-11T09:28:45.128 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rgw_sync": "1/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_rocksdb": "4/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_backref": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_cache": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_cleaner": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_device": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_epm": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_journal": "0/5", 2024-09-11T09:28:45.129 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_lba": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_odata": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_omap": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_onode": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_t": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_seastore_tm": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_striper": "0/1", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_test": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_throttle": "1/1", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_timer": "0/1", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_tp": "0/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "debug_trace": "1/5", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-11T09:28:45.130 INFO:teuthology.orchestra.run.smithi017.stdout: "device_failure_prediction_mode": "none", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_graylog": "false", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_journald": "false", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_stderr": "true", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "err_to_syslog": "false", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event_tracing": "false", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_addr": "0.0.0.0", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_http_port": "9926", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_prio_limit": "5", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-11T09:28:45.131 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_sort_metrics": "true", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "exporter_stats_period": "5", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "fake_statfs_for_testing": "0", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "fatal_signal_handlers": "true", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filer_max_purge_ops": "10", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filer_max_truncate_ops": "128", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_apply_finisher_threads": "1", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_blackhole": "false", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_btrfs_clone_range": "true", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_btrfs_snap": "true", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_caller_concurrency": "10", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_collect_device_partition_information": "true", 2024-09-11T09:28:45.132 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_commit_timeout": "600.000000", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_inject_read_err": "false", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_omap_check": "false", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_debug_verify_split": "false", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_dump_file": "", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fadvise": "true", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fail_eio": "true", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fd_cache_shards": "16", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fd_cache_size": "128", 2024-09-11T09:28:45.133 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fiemap": "false", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fiemap_threshold": "4096", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_inject_stall": "0", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_journal_parallel": "false", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_journal_trailing": "false", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_journal_writeahead": "false", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_kill_at": "0", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-11T09:28:45.134 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs": "0", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size": "0", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_merge_threshold": "-10", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-11T09:28:45.135 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_odsync_write": "false", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_omap_backend": "rocksdb", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_omap_backend_path": "", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_op_thread_timeout": "60", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_op_threads": "2", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_punch_hole": "false", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-11T09:28:45.136 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_queue_max_ops": "50", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_seek_data_hole": "false", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_sloppy_crc": "false", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_splice": "false", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_split_multiple": "2", 2024-09-11T09:28:45.137 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_split_rand_factor": "20", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_update_to": "1000", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_enable": "true", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-11T09:28:45.138 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_xfs_extsize": "false", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "filestore_zfs_snap": "false", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fio_dir": "/tmp/fio", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "23a8bf95-caba-4fea-a762-61040a1c1b77", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_allow_other": "true", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_atomic_o_trunc": "true", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_big_writes": "true", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_debug": "false", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_default_permissions": "false", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_disable_pagecache": "false", 2024-09-11T09:28:45.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_max_write": "0", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_multithreaded": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_require_active_mds": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_set_user_groups": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_splice_move": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_splice_read": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_splice_write": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "fuse_use_invalidate_cb": "true", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "gss_target_name": "ceph", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_file": "", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_inject_failure": "0", 2024-09-11T09:28:45.140 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_interval": "5", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "host": "smithi017", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-11T09:28:45.141 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "inject_early_sigterm": "false", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "jaeger_agent_port": "6799", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "jaeger_tracing_enable": "false", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_aio": "true", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_align_min_size": "65536", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_block_align": "true", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_block_size": "4096", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_dio": "true", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_discard": "false", 2024-09-11T09:28:45.142 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_force_aio": "false", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_ignore_corruption": "false", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_max_write_bytes": "10485760", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_max_write_entries": "100", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_replay_from": "0", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_write_header_frequency": "0", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journal_zero_on_create": "false", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journaler_prefetch_periods": "10", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journaler_prezero_periods": "5", 2024-09-11T09:28:45.143 INFO:teuthology.orchestra.run.smithi017.stdout: "journaler_write_head_interval": "15", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "keyfile": "", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_backend": "rocksdb", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_default_stripe_size": "65536", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_fsck_on_mount": "false", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_max_bytes": "67108864", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_max_ops": "512", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_nid_prealloc": "1024", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_onode_map_size": "1024", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-11T09:28:45.144 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_sync_submit_transaction": "false", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "kstore_sync_transaction": "false", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "librados_thread_count": "2", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "lockdep": "false", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "lockdep_force_backtrace": "false", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_coarse_timestamps": "true", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_flush_on_exit": "false", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_graylog_host": "127.0.0.1", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_graylog_port": "12201", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_max_new": "1000", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_max_recent": "10000", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_stderr_prefix": "", 2024-09-11T09:28:45.145 INFO:teuthology.orchestra.run.smithi017.stdout: "log_stop_at_utilization": "0.970000", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_file": "true", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_graylog": "false", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_journald": "false", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_stderr": "false", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "log_to_syslog": "false", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "max_rotating_auth_attempts": "10", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_action_on_write_error": "1", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_alternate_name_max": "8192", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_asio_thread_count": "2", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_export_pin": "true", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_dirs": "true", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-11T09:28:45.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_interval": "5", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_interval": "10", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_max": "-1", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_max_until": "-1", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_merge_size": "50", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_midchunk": "0.300000", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_min_start": "0.200000", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_minchunk": "0.001000", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_mode": "0", 2024-09-11T09:28:45.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_need_max": "1.200000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_need_min": "0.800000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_overload_epochs": "2", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_bits": "3", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_size": "10000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_target_decay": "10.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_beacon_grace": "15.000000", 2024-09-11T09:28:45.148 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_beacon_interval": "4.000000", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_mid": "0.700000", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_quiesce_delay": "0", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_release_free_interval": "10", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_reservation": "0.050000", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-11T09:28:45.149 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_trim_interval": "1", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache_trim_threshold": "262144", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_client_prealloc_inos": "1000", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_connect_bootstrapping": "false", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_damage_table_max_entries": "10000", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_auth_pins": "false", 2024-09-11T09:28:45.150 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_frag": "false", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_scatterstat": "false", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_debug_subtrees": "false", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_decay_halflife": "5.000000", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_default_dir_hash": "2", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_defer_session_stale": "true", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_deny_all_reconnect": "false", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_keys_per_op": "16384", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_max_commit_size": "10", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_max_entries": "0", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dir_prefetch": "true", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_on_map": "false", 2024-09-11T09:28:45.151 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_early_reply": "true", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_enable_op_tracker": "true", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_enforce_unique_name": "true", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_random": "true", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-11T09:28:45.152 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_health_summarize_threshold": "10", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_health_dummy": "false", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_migrator_session_race": "false", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-11T09:28:45.153 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_join_fs": "", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_journal_format": "1", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_create_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_export_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_import_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_journal_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_journal_expire_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_journal_replay_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_link_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_mdstable_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_openc_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_rename_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_kill_shutdown_at": "0", 2024-09-11T09:28:45.154 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_event_large_threshold": "524288", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_events_per_segment": "1024", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_max_events": "-1", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_max_segments": "128", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_pause": "false", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_segment_size": "0", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_trim_threshold": "128", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log_warn_factor": "2.000000", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_caps_per_client": "1048576", 2024-09-11T09:28:45.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_completed_flushes": "100000", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_completed_requests": "100000", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_export_size": "20971520", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_file_recover": "32", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_purge_files": "64", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_purge_ops": "8192", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_max_snaps_per_dir": "100", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_metrics_update_interval": "2", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_min_caps_per_client": "100", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_min_caps_working_set": "10000", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-11T09:28:45.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_numa_node": "-1", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_complaint_time": "30.000000", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_history_duration": "600", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_history_size": "20", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_history_slow_op_size": "20", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_op_log_threshold": "5", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_ping_grace": "15", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_ping_interval": "5", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_max_caps": "30000", 2024-09-11T09:28:45.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_recall_warning_threshold": "262144", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_replay_interval": "1.000000", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_root_ino_gid": "0", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_root_ino_uid": "0", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-11T09:28:45.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_shutdown_check": "0", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_skip_ino": "0", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_snap_max_uid": "4294967294", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_snap_min_uid": "0", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_snap_rstat": "false", 2024-09-11T09:28:45.159 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_standby_replay_damaged": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_symlink_recovery": "true", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_thrash_exports": "0", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_thrash_fragments": "0", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_tick_interval": "5.000000", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_valgrind_exit": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_verify_backtrace": "1", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_verify_scatter": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_wipe_sessions": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool_debug": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-11T09:28:45.160 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_device_bytes": "1073741824", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_page_set": "false", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "memstore_page_size": "65536", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_client_bytes": "134217728", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_client_messages": "512", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_disabled_modules": "", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_enable_op_tracker": "true", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-11T09:28:45.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_max_pg_num_change": "128", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mds_bytes": "134217728", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mds_messages": "128", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mon_bytes": "134217728", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_mon_messages": "128", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_op_history_duration": "600", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_op_history_size": "20", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_op_log_threshold": "5", 2024-09-11T09:28:45.162 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_osd_bytes": "536870912", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_osd_messages": "8192", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_pool": "true", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_standby_modules": "true", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_stats_period": "5", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_stats_threshold": "5", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_tick_period": "2", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_allow_pool_delete": "true", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_allow_pool_size_one": "true", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_auth_validate_all_caps": "true", 2024-09-11T09:28:45.163 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_bytes": "104857600", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_directed_command_retry": "5", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_hunt_parallel": "3", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_log_interval": "1.000000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_ping_interval": "10.000000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-11T09:28:45.164 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_client_target_rank": "-1", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_file_level": "debug", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_file": "true", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_journald": "false", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-11T09:28:45.165 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_compact_on_bootstrap": "false", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_compact_on_start": "false", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_compact_on_trim": "true", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_cpu_threads": "4", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_crush_min_required_version": "hammer", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_daemon_bytes": "419430400", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data_avail_crit": "5", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data_avail_warn": "5", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_data_size_warn": "16106127360", 2024-09-11T09:28:45.166 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_dump_json": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_dump_transactions": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_extra_checks": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_require_reef": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_no_require_squid": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-11T09:28:45.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_down_mkfs_grace": "120", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_down_uptime_grace": "60", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_election_default_strategy": "1", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_election_timeout": "5.000000", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_elector_ping_divisor": "2", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_enable_op_tracker": "true", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_fake_pool_delete": "false", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_force_quorum_join": "false", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_globalid_prealloc": "10000", 2024-09-11T09:28:45.168 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_detail_to_clog": "false", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_log_update_period": "5", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_max_detail": "50", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_to_clog": "true", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_to_clog_interval": "600", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_host": "172.21.15.17,[v2:172.21.15.17:3301,v1:172.21.15.17:6790],172.21.15.96", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_host_override": "", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_initial_members": "", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-11T09:28:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_lease": "5.000000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_log_full_interval": "50", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_log_max": "10000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_log_max_summary": "50", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_log_epochs": "500", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_osd": "10000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_pg_per_osd": "10000", 2024-09-11T09:28:45.170 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_pool_pg_num": "65536", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mds_force_trim_to": "0", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mds_skip_sanity": "false", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_memory_autotune": "true", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_memory_target": "2147483648", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_beacon_grace": "30", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_digest_period": "5", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_inactive_grace": "60", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-11T09:28:45.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_min_osdmap_epochs": "500", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_complaint_time": "30", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_duration": "600", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_size": "20", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_slow_op_size": "20", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_op_log_threshold": "5", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_auto_mark_in": "false", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-11T09:28:45.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_cache_size": "500", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_down_out_interval": "600", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_force_trim_to": "0", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-11T09:28:45.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_min_down_reporters": "2", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-11T09:28:45.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_report_timeout": "900", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_stuck_threshold": "60", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-11T09:28:45.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_probe_timeout": "2.000000", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_max_change": "0.050000", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_max_osds": "4", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-11T09:28:45.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_interval": "86400", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_max_keys": "100", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_scrub_timeout": "300", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_session_timeout": "300", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_smart_report_timeout": "5", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stat_smooth_intervals": "6", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_pool_min_size": "2", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_pool_size": "4", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_debug": "false", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-11T09:28:45.177 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_provider_kill_at": "0", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_requester_kill_at": "0", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_sync_timeout": "60.000000", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_target_pg_per_osd": "100", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_tick_interval": "5", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_timecheck_interval": "300.000000", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_use_min_delay_socket": "false", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_older_version_delay": "604800", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-11T09:28:45.178 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_misplaced": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_older_version": "true", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-11T09:28:45.179 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": "", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-11T09:28:45.180 INFO:teuthology.orchestra.run.smithi017.stdout: "motr_tracing_enabled": "false", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_op_threads": "3", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_cm": "false", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_device_name": "", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_dscp": "96", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_local_gid": "", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_port_num": "1", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-11T09:28:45.181 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_sl": "3", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_support_srq": "true", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_rdma_type": "ib", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_async_reap_threshold": "5", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_before_connect": "false", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_ipv4": "true", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_ipv6": "false", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_msgr1": "true", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_msgr2": "true", 2024-09-11T09:28:45.182 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_port_max": "7568", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_port_min": "6800", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_retry_count": "3", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_bind_retry_delay": "5", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_client": "false", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_mds": "false", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_mgr": "false", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_mon": "false", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_blackhole_osd": "false", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_client_mode": "crc secure", 2024-09-11T09:28:45.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_cluster_mode": "crc secure", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_cluster_type": "", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_compress_secure": "false", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_connection_idle_timeout": "900", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_connection_ready_timeout": "10", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_crc_data": "true", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_crc_header": "true", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_bad_msg": "false", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_bug": "true", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_old_message": "true", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_skipped_message": "false", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-11T09:28:45.184 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_coremask": "0xF", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_enable_tso": "true", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_hugepages": "", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_lro": "true", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_memory_channel": "4", 2024-09-11T09:28:45.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_pmd": "", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_port_id": "0", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_dump_on_send": "false", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_initial_backoff": "0.200000", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_delay_max": "1.000000", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_delay_type": "", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_network_congestion": "0", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_inject_socket_failures": "5000", 2024-09-11T09:28:45.186 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_learn_addr_from_peer": "true", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_max_accept_failures": "4", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_max_backoff": "15.000000", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_mon_client_mode": "secure crc", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_mon_service_mode": "secure crc", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_osd_compress_min_size": "1024", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_osd_compress_mode": "none", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_pq_min_cost": "65536", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_public_type": "", 2024-09-11T09:28:45.187 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_service_mode": "crc secure", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_listen_backlog": "512", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_nodelay": "true", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_tcp_rcvbuf": "0", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "ms_type": "async+posix", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "no_config_file": "false", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_completion_locks_per_session": "32", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_inflight_ops": "1024", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-11T09:28:45.188 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_tick_interval": "5.000000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter_timeout": "10.000000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "objectstore_blackhole": "false", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "openssl_engine_opts": "", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_delay_time": "5.000000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_hist_halflife": "1000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_max_low_ops": "2", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_max_ops": "4", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_agent_slop": "0.020000", 2024-09-11T09:28:45.189 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_asio_thread_count": "2", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_async_recovery_min_cost": "100", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backfill_scan_max": "512", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backfill_scan_min": "64", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backoff_on_degraded": "false", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backoff_on_peering": "false", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_backoff_on_unfound": "true", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_beacon_report_interval": "300", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_duration": "30", 2024-09-11T09:28:45.190 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_max_block_size": "67108864", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_blkin_trace_all": "false", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-11T09:28:45.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_check_for_log_corruption": "false", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_class_update_on_start": "true", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_message_cap": "256", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_message_size_cap": "524288000", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_op_priority": "63", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_client_watch_timeout": "30", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_command_max_records": "256", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_compact_on_start": "false", 2024-09-11T09:28:45.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_update_on_start": "true", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_crush_update_weight_set": "true", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_feed_pullee": "-1", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-11T09:28:45.193 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_misdirected_ops": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_no_acting_change": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_no_purge_strays": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_op_order": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_shutdown": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-11T09:28:45.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_trim_objects": "false", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_snaps": "false", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_keys": "1024", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_stride": "524288", 2024-09-11T09:28:45.195 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_default_notify_timeout": "30", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep": "0.000000", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_discard_disconnected_ops": "true", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_enable_op_tracker": "true", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-11T09:28:45.196 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_info": "true", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_shutdown": "true", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_function_tracing": "false", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_grace": "20", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_interval": "6", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-11T09:28:45.197 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_min_peers": "10", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_min_size": "2000", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_stale": "600", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_hit_set_max_size": "100000", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_hit_set_min_size": "1000", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-11T09:28:45.198 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_journal_size": "5120", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_kill_backfill_at": "0", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_cache_size": "50", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_dedup": "true", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_message_max": "40", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_map_share_max_epochs": "40", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_attr_name_len": "100", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_attr_size": "0", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_backfills": "1", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_markdown_count": "5", 2024-09-11T09:28:45.199 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_markdown_period": "600", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_object_name_len": "2048", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_object_namespace_len": "256", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_object_size": "134217728", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pg_blocked_by": "16", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pg_log_entries": "10000", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_pgls": "1024", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_push_cost": "8388608", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_push_objects": "10", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_scrubs": "3", 2024-09-11T09:28:45.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_trimming_pgs": "2", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_write_op_reply_len": "64", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_max_write_size": "90", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-11T09:28:45.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_profile": "balanced", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-11T09:28:45.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_base": "805306368", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_cache_min": "134217728", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_target": "4294967296", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_target_autotune": "false", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_min_pg_log_entries": "250", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_min_recovery_priority": "0", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_report_interval": "5", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-11T09:28:45.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_num_cache_shards": "32", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_num_op_tracker_shard": "32", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_numa_auto_affinity": "true", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_numa_node": "-1", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_numa_prefer_iface": "true", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objecter_finishers": "1", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objectstore": "bluestore", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objectstore_fuse": "false", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_objectstore_tracing": "false", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_complaint_time": "30.000000", 2024-09-11T09:28:45.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_duration": "600", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_size": "20", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_slow_op_size": "20", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_log_threshold": "5", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_shards": "0", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_shards_hdd": "5", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_shards_ssd": "8", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-11T09:28:45.205 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_pq_min_cost": "65536", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_queue_cut_off": "high", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_op_thread_timeout": "15", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_open_classes_on_start": "true", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_os_flags": "0", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_peering_op_priority": "255", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_delete_cost": "1048576", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_delete_priority": "5", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-11T09:28:45.206 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_log_trim_max": "10000", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_log_trim_min": "100", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_crimson": "false", 2024-09-11T09:28:45.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_flags": "0", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_min_size": "0", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_pg_num": "32", 2024-09-11T09:28:45.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_pgp_num": "0", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_read_ratio": "70", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_size": "2", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_default_type": "replicated", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_push_per_object_cost": "1000", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recover_clone_overlap": "true", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_cost": "20971520", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_active": "0", 2024-09-11T09:28:45.209 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_max_single_start": "1", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_op_priority": "3", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_priority": "5", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep": "0.000000", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_repair_during_recovery": "false", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_requested_scrub_priority": "120", 2024-09-11T09:28:45.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_auto_repair": "false", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_begin_hour": "0", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_begin_week_day": "0", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_chunk_max": "25", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_chunk_min": "5", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_cost": "52428800", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_during_recovery": "false", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_end_hour": "0", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_end_week_day": "0", 2024-09-11T09:28:45.211 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_event_cost": "4096", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_invalid_stats": "true", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_max_preemptions": "5", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_priority": "5", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_scrub_sleep": "0.000000", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-11T09:28:45.212 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_skip_data_digest": "false", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_smart_report_timeout": "5", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_cost": "1048576", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_priority": "5", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-11T09:28:45.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_target_transaction_size": "30", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_tracing": "false", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_use_stale_snap": "false", 2024-09-11T09:28:45.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "osdc_blkin_trace_all": "false", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_kill_at": "0", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_max_join_drift": "10", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_min": "500", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_min_wait": "0.050000", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_propose_interval": "1.000000", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_service_trim_max": "500", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_service_trim_min": "250", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_stash_full_interval": "25", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_trim_max": "500", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "paxos_trim_min": "250", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "perf": "true", 2024-09-11T09:28:45.215 INFO:teuthology.orchestra.run.smithi017.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "-", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrv": "", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_bind_addr": "-", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_network": "", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_network_interface": "", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "qat_compressor_busy_polling": "false", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "qat_compressor_enabled": "false", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "qat_compressor_session_max_number": "256", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "rados_mon_op_timeout": "0", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "rados_osd_op_timeout": "0", 2024-09-11T09:28:45.216 INFO:teuthology.orchestra.run.smithi017.stdout: "rados_tracing": "false", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_atime_update_interval": "60", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_balance_parent_reads": "false", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_balance_snap_reads": "false", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_blkin_trace_all": "false", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache": "true", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-11T09:28:45.217 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_policy": "writearound", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_size": "33554432", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_clone_copy_on_read": "false", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_compression_hint": "none", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_concurrent_management_ops": "10", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_clone_format": "auto", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_data_pool": "", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_features": "61", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_format": "2", 2024-09-11T09:28:45.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_map_options": "", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_order": "22", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_pool": "rbd", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_stripe_count": "0", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_default_stripe_unit": "0", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_enable_alloc_hint": "true", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_io_scheduler": "simple", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-11T09:28:45.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_order": "24", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_pool": "", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_journal_splay_width": "4", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_localize_parent_reads": "false", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_localize_snap_reads": "false", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-11T09:28:45.220 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-11T09:28:45.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-11T09:28:45.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_mtime_update_interval": "60", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_non_blocking_aio": "true", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_op_thread_timeout": "60", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_op_threads": "1", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_parent_cache_enabled": "false", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-11T09:28:45.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_plugins": "", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_bps_burst": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_bps_limit": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_exclude_ops": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_iops_burst": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_iops_limit": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-11T09:28:45.224 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_read_from_replica_policy": "default", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-11T09:28:45.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_skip_partial_discard": "true", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_tracing": "false", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_validate_names": "true", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd_validate_pool": "true", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "restapi_base_url": "", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "restapi_log_level": "", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-11T09:28:45.226 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_acl_grants_max_num": "100", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_admin_entry": "admin", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_backend_store": "rados", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_barbican_url": "", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_beast_enable_async": "true", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_counters_cache": "false", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-11T09:28:45.227 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cache_enabled": "true", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cache_expiry_interval": "900", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cache_lru_size": "25000", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_config_store": "rados", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_content_length_compat": "false", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_copy_obj_progress": "true", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cors_rules_max_num": "100", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_cross_domain_policy": "", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_addr": "", 2024-09-11T09:28:45.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_password": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_kmip_username": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_require_ssl": "true", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-11T09:28:45.229 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_addr": "", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_auth": "token", 2024-09-11T09:28:45.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_namespace": "", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_prefix": "", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_token_file": "", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_buffersize": "524288", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_low_speed_time": "300", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-11T09:28:45.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_d4n_port": "6379", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_changes_size": "1000", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_num_shards": "128", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-11T09:28:45.232 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_log_window": "30", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-11T09:28:45.233 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-11T09:28:45.234 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dns_name": "", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dns_s3website_name": "", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_dynamic_resharding": "true", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_gc_threads": "true", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_lc_threads": "true", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_ops_log": "false", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_quota_threads": "true", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_static_website": "false", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enable_usage_log": "false", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_enforce_swift_acls": "true", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_exit_timeout_secs": "120", 2024-09-11T09:28:45.235 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_expose_bucket": "false", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_extended_http_attrs": "", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_filter": "none", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_frontends": "beast port=7480", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_deferred": "50", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_objs": "32", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-11T09:28:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_gc_processor_period": "3600", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_init_timeout": "300", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_kafka_connection_idle": "300", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-11T09:28:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_domain": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_password": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_password_path": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_project": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_tenant": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_token": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_token_path": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_admin_user": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_api_version": "2", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_domain": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_password": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_project": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-11T09:28:45.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_barbican_user": "", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_url": "", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_debug_interval": "-1", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_lock_max_time": "90", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_objs": "32", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_rules": "1000", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_worker": "3", 2024-09-11T09:28:45.239 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lc_thread_delay": "0", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_dnattr": "uid", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_searchfilter": "", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ldap_uri": "", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_http_headers": "", 2024-09-11T09:28:45.240 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_log_object_name_utc": "false", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_attr_name_len": "0", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_attr_size": "0", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_chunk_size": "4194304", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-11T09:28:45.241 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_listing_results": "1000", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_notify_retries": "10", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_put_param_size": "1048576", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_put_size": "5368709120", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_max_slo_entries": "1000", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_md_log_max_shards": "64", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-11T09:28:45.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_mp_lock_max_time": "600", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_max_gc": "300", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-11T09:28:45.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_num_async_rados_threads": "32", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_num_control_oids": "8", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_numa_node": "-1", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_objexp_chunk_size": "100", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_objexp_gc_interval": "600", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-11T09:28:45.244 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_op_thread_timeout": "600", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_op_tracing": "false", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_opa_token": "", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_opa_url": "", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_opa_verify_ssl": "true", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_rados": "false", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_ops_log_socket_path": "", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_push_interval": "2.000000", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-11T09:28:45.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_posix_cache_lanes": "3", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_posix_cache_partitions": "3", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_print_continue": "true", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-11T09:28:45.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rados_tracing": "false", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_realm": "", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_realm_id": "", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_region": "", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_request_uri": "", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_batch_size": "64", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_max_aio": "128", 2024-09-11T09:28:45.247 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_num_logs": "16", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_reshard_thread_interval": "600", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_resolve_cname": "false", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_run_sync_thread": "true", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-11T09:28:45.248 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_scheduler_type": "throttler", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_script_uri": "", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_service_provider_name": "", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_client_id": "", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_client_secret": "", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_entry": "sts", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_key": "sts", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_min_session_duration": "900", 2024-09-11T09:28:45.249 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sts_token_introspection_url": "", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_account_in_url": "false", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_auth_entry": "auth", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_auth_url": "", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_custom_header": "", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_need_stats": "true", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_tenant_name": "", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_token_expiration": "86400", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_url": "", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_url_prefix": "swift", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-11T09:28:45.250 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_lease_period": "120", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_thread_pool_size": "512", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-11T09:28:45.251 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_comment": "", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_createby": "", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_encoding": "", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_flag": "false", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_origin": "", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_torrent_tracker": "", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_trust_forwarded_https": "false", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-11T09:28:45.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_max_shards": "32", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_usage_max_user_shards": "1", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_use_opa_authz": "false", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_counters_cache": "false", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_max_buckets": "1000", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_policies_max_num": "100", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-11T09:28:45.253 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_user_unique_email": "true", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_verify_ssl": "true", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zone": "", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zone_id": "", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zonegroup": "", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zonegroup_id": "", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-11T09:28:45.254 INFO:teuthology.orchestra.run.smithi017.stdout: "rgwlc_auto_session_clear": "true", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_block_size": "4096", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_size": "536870912", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-11T09:28:45.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_index_type": "binary_search", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_partition_filters": "false", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_perf": "false", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-11T09:28:45.256 INFO:teuthology.orchestra.run.smithi017.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "run_dir": "/var/run/ceph", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_block_create": "true", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_cache_lru_size": "67108864", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_default_max_object_size": "16777216", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_device_size": "53687091200", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_full_integrity_check": "false", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_batch_capacity": "16", 2024-09-11T09:28:45.257 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_main_device_type": "SSD", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "seastore_segment_size": "67108864", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "set_keepcaps": "false", 2024-09-11T09:28:45.258 INFO:teuthology.orchestra.run.smithi017.stdout: "setgroup": "", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "setuser": "", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "setuser_match_path": "", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "thp": "false", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "threadpool_default_timeout": "60", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout: "throttler_perf_counter": "true" 2024-09-11T09:28:45.259 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:45.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-11T09:28:45.377 INFO:teuthology.orchestra.run.smithi017.stdout:mon.a: {} 2024-09-11T09:28:45.378 INFO:teuthology.orchestra.run.smithi017.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:28:45.387 INFO:teuthology.orchestra.run.smithi017.stdout:mon.b: {} 2024-09-11T09:28:45.388 INFO:teuthology.orchestra.run.smithi017.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:28:45.396 INFO:teuthology.orchestra.run.smithi017.stdout:mon.c: {} 2024-09-11T09:28:45.396 INFO:teuthology.orchestra.run.smithi017.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:28:45.407 DEBUG:teuthology.orchestra.run.smithi017:> type ceph-objectstore-tool 2024-09-11T09:28:45.421 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T09:28:45.422 DEBUG:teuthology.orchestra.run.smithi096:> type ceph-objectstore-tool 2024-09-11T09:28:45.441 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T09:28:45.441 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-11T09:28:45.452 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-11T09:28:45.452 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-11T09:28:45.452 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:28:45.452 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:28:45.452 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-11T09:28:45.452 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T09:28:45.453 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T09:28:45.453 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-11T09:28:45.453 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-11T09:28:45.453 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-11T09:28:45.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:28:45.454 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-11T09:28:45.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:28:45.454 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-11T09:28:45.455 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:28:45.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-11T09:28:45.482 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:45.482+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:45.521 INFO:teuthology.orchestra.run.smithi107.stdout:mapped writes DISABLED 2024-09-11T09:28:45.524 INFO:teuthology.orchestra.run.smithi107.stdout:mapped reads DISABLED 2024-09-11T09:28:45.524 INFO:teuthology.orchestra.run.smithi107.stdout:Seed set to 4282 2024-09-11T09:28:45.524 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd mode enabled 2024-09-11T09:28:45.555 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:45.556+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:45.582 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:45.582 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T09:28:45.583 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:45.597 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:45.597 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:45.597 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:45.597 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:45.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:28:45.607 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:45.607 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:45.640 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:45.640 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:45.663 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:45.663 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:45.663 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:45.664+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:45.696 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:45.696 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:45.729 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:45.730 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:45.754 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:45.754 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:45.757 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:45.757 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:45.757 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:45.757+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:45.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:28:45.858 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:45.858+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:45.916 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:45.916 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:45.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:28:45.968 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:45.969+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:46.032 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.032 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.041 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:28:46.059 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:46.060+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:46.160 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:46.161+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:46.187 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.187 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.196 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:28:46.261 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:46.262+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:46.344 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.344 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:28:46.362 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:46.362+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:46.459 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.459 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.463 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:46.463+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:46.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:28:46.564 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:46.564+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:46.575 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.575 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.584 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:28:46.668 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:46.669+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:46.692 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:28:46.740 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.740 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:28:46.790 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:46.791+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:46.860 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.860 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.866 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:46.867+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:46.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:28:46.967 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:46.967+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:46.970 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:46.971 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:46.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:28:47.068 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:47.069+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:47.121 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.121 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.130 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:28:47.175 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:47.175+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:47.232 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.232 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.241 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:28:47.270 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:47.271+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:47.372 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:47.372+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:47.384 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.384 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.393 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:28:47.472 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:47.472+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:47.494 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.494 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:28:47.573 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:47.574+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:47.644 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.644 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:28:47.697 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:47.698+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:47.758 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.758 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:28:47.769 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:28:47.808 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:47.808+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:47.875 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:47.876+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:47.904 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:47.905 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:47.910 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:47.910 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:47.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:28:47.922 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:47.922 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:47.938 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:47.939 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:47.956 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:47.956 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:47.973 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:47.973 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:47.976 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:47.976+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:47.990 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:47.990 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:48.029 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:48.029 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:48.029 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:48.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:28:48.078 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:48.079+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:48.178 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:48.178+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:48.184 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:48.185 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:48.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:28:48.278 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:48.279+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:20 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'10, mlcod=14'8)", 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.821458+0000", 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.521037528000001, 2024-09-11T09:28:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010219389999999999, 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821458+0000", 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821458+0000", 2024-09-11T09:28:48.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821459+0000", 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1699999999999997e-07 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821576+0000", 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117272 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821590+0000", 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3729e-05 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821596+0000", 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6099999999999997e-06 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821650+0000", 2024-09-11T09:28:48.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4169999999999998e-05 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.821666+0000", 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5950000000000001e-05 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.822439+0000", 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00077287299999999998 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.822480+0000", 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1418000000000002e-05 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:31 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'17, mlcod=14'14)", 2024-09-11T09:28:48.348 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.831219+0000", 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.511276909999999, 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001025593, 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831219+0000", 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831219+0000", 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831220+0000", 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6199999999999996e-07 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831238+0000", 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8538000000000001e-05 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831252+0000", 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3396e-05 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831263+0000", 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1704e-05 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831323+0000", 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9225000000000003e-05 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831346+0000", 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3383e-05 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.832201+0000", 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00085521800000000004 2024-09-11T09:28:48.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.832244+0000", 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3266999999999998e-05 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:32 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'18, mlcod=14'14)", 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.831714+0000", 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.510781864999998, 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001053579, 2024-09-11T09:28:48.354 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831714+0000", 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831714+0000", 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831714+0000", 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5599999999999995e-07 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831786+0000", 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1613999999999996e-05 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831797+0000", 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0664e-05 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831802+0000", 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.429e-06 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831841+0000", 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9326000000000003e-05 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.831863+0000", 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1501999999999998e-05 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.832726+0000", 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00086308399999999996 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.832767+0000", 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1403999999999999e-05 2024-09-11T09:28:48.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:35 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'21, mlcod=14'17)", 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.508564863, 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010916070000000001, 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.359 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:48.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1399999999999997e-07 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833998+0000", 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6241000000000005e-05 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834010+0000", 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2349e-05 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834025+0000", 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4962e-05 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834063+0000", 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7855000000000003e-05 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834076+0000", 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3352e-05 2024-09-11T09:28:48.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834987+0000", 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00091061400000000002 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.835022+0000", 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5620000000000001e-05 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:38 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'24, mlcod=14'22)", 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.836461+0000", 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.506034923000001, 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010471429999999999, 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836461+0000", 2024-09-11T09:28:48.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836461+0000", 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836462+0000", 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9299999999999997e-07 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836497+0000", 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5753999999999997e-05 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836510+0000", 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2815e-05 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836516+0000", 2024-09-11T09:28:48.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.429e-06 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836562+0000", 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6582000000000002e-05 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.836582+0000", 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.969e-05 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.837472+0000", 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00088973600000000002 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.837508+0000", 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6344000000000003e-05 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.368 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:54 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'32, mlcod=14'29)", 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.848156+0000", 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.494339712999999, 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010451620000000001, 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848156+0000", 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848156+0000", 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848157+0000", 2024-09-11T09:28:48.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3300000000000001e-07 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848162+0000", 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848174+0000", 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1980999999999999e-05 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848181+0000", 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7630000000000003e-06 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848227+0000", 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6581e-05 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848250+0000", 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2603e-05 2024-09-11T09:28:48.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.849154+0000", 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090424899999999998 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.849201+0000", 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7168e-05 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:55 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'33, mlcod=14'29)", 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.848574+0000", 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.493922067, 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001007162, 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848574+0000", 2024-09-11T09:28:48.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848574+0000", 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848574+0000", 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6899999999999997e-07 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848643+0000", 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8876999999999998e-05 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848653+0000", 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7829999999999993e-06 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848658+0000", 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6870000000000002e-06 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848710+0000", 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2219999999999998e-05 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.848733+0000", 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3544e-05 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.849541+0000", 2024-09-11T09:28:48.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00080782 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.849581+0000", 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9662000000000001e-05 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:57 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'35, mlcod=14'33)", 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.852196+0000", 2024-09-11T09:28:48.378 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.490299703000002, 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010450069999999999, 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852196+0000", 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852196+0000", 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852197+0000", 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8899999999999999e-07 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852229+0000", 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2591e-05 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852240+0000", 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0879e-05 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852253+0000", 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2584e-05 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852307+0000", 2024-09-11T09:28:48.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4630999999999999e-05 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.852331+0000", 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.408e-05 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.853203+0000", 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00087129199999999996 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.853241+0000", 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8361000000000001e-05 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:67 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'41, mlcod=14'39)", 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.859515+0000", 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.482980834999999, 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00098422599999999994, 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859515+0000", 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859515+0000", 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859517+0000", 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.697e-06 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859528+0000", 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1316e-05 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859555+0000", 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6874000000000001e-05 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859567+0000", 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2e-05 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859670+0000", 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103732 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859705+0000", 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4310000000000002e-05 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860453+0000", 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00074833400000000002 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860499+0000", 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5963e-05 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'42, mlcod=14'39)", 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.859822+0000", 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.48267367, 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011026440000000001, 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.390 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859822+0000", 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859822+0000", 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859823+0000", 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1659999999999999e-06 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859845+0000", 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1654e-05 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859879+0000", 2024-09-11T09:28:48.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3936000000000002e-05 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859883+0000", 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7629999999999998e-06 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860094+0000", 2024-09-11T09:28:48.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000211719 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860116+0000", 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1594e-05 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860895+0000", 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00077924899999999998 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860925+0000", 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9563000000000001e-05 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:87 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'57, mlcod=14'53)", 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.879493+0000", 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.463003024999999, 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010545120000000001, 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.395 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879493+0000", 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879493+0000", 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879493+0000", 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8100000000000002e-07 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879552+0000", 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8291000000000001e-05 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879561+0000", 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2650000000000002e-06 2024-09-11T09:28:48.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879567+0000", 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879613+0000", 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5977999999999999e-05 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.879626+0000", 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2595999999999999e-05 2024-09-11T09:28:48.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.880513+0000", 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00088674299999999995 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.880547+0000", 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4643e-05 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.399 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:105 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'67, mlcod=14'64)", 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.022096+0000", 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.320399962, 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00098235100000000001, 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022096+0000", 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022096+0000", 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022096+0000", 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5400000000000002e-07 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022100+0000", 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8039999999999999e-06 2024-09-11T09:28:48.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022107+0000", 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9619999999999996e-06 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022112+0000", 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1699999999999996e-06 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022154+0000", 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2308000000000002e-05 2024-09-11T09:28:48.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.022173+0000", 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8434e-05 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.023040+0000", 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00086677800000000001 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.023078+0000", 2024-09-11T09:28:48.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8441e-05 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:124 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'86, mlcod=14'84)", 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.421352+0000", 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.921143369999999, 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00093463200000000002, 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421352+0000", 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421352+0000", 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421355+0000", 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:28:48.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421377+0000", 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2093e-05 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421407+0000", 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0239999999999998e-05 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421421+0000", 2024-09-11T09:28:48.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3849000000000001e-05 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421537+0000", 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011647299999999999 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.421578+0000", 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1136000000000001e-05 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.422256+0000", 2024-09-11T09:28:48.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00067789300000000005 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.422287+0000", 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0651000000000002e-05 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:142 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'100, mlcod=14'98)", 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.433206+0000", 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.909289325, 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00093658499999999998, 2024-09-11T09:28:48.408 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433206+0000", 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433206+0000", 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433207+0000", 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8199999999999999e-07 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433273+0000", 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6400000000000001e-05 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433283+0000", 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4590000000000004e-06 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433288+0000", 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433343+0000", 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5325999999999998e-05 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433360+0000", 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6595999999999998e-05 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434107+0000", 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00074694599999999996 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434143+0000", 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6171000000000003e-05 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:143 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'101, mlcod=14'98)", 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.433601+0000", 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.908895008000002, 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001044775, 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433601+0000", 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433601+0000", 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433601+0000", 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2099999999999996e-07 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433664+0000", 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2644999999999994e-05 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433675+0000", 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0754000000000001e-05 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433684+0000", 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0159999999999997e-06 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433740+0000", 2024-09-11T09:28:48.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6492000000000001e-05 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.433762+0000", 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1522000000000002e-05 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434610+0000", 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00084862100000000003 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434645+0000", 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5104000000000001e-05 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:144 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'102, mlcod=14'98)", 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.434099+0000", 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.908396234000001, 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00096321600000000003, 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.417 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434099+0000", 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434099+0000", 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434100+0000", 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7400000000000002e-07 2024-09-11T09:28:48.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434173+0000", 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3025e-05 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434183+0000", 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2329999999999995e-06 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434189+0000", 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5679999999999996e-06 2024-09-11T09:28:48.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434234+0000", 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5000000000000003e-05 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.434254+0000", 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0166999999999999e-05 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.435024+0000", 2024-09-11T09:28:48.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00077008999999999997 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.435063+0000", 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8359000000000003e-05 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:148 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'106, mlcod=14'104)", 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.845335+0000", 2024-09-11T09:28:48.421 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.497160694000002, 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00093097199999999996, 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845335+0000", 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845335+0000", 2024-09-11T09:28:48.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845337+0000", 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.137e-06 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845354+0000", 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6595999999999998e-05 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845386+0000", 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1933000000000001e-05 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845401+0000", 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5553000000000002e-05 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845560+0000", 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000158352 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.845589+0000", 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9490999999999999e-05 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.846234+0000", 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00064464800000000005 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.846266+0000", 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2261999999999997e-05 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:155 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'109, mlcod=14'105)", 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.849007+0000", 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.493488314, 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010165510000000001, 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849007+0000", 2024-09-11T09:28:48.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849007+0000", 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849008+0000", 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4000000000000002e-07 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849097+0000", 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8707000000000003e-05 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849106+0000", 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2329999999999995e-06 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849112+0000", 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5969999999999997e-06 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849146+0000", 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3411999999999998e-05 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849159+0000", 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3429e-05 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.849977+0000", 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00081822399999999999 2024-09-11T09:28:48.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850024+0000", 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6508999999999998e-05 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:158 1.0 e15/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'112, mlcod=15'107)", 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.850666+0000", 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.491829407000001, 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001028897, 2024-09-11T09:28:48.430 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850666+0000", 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850666+0000", 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850667+0000", 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850740+0000", 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3632000000000002e-05 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850750+0000", 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7340000000000003e-06 2024-09-11T09:28:48.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850756+0000", 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.682e-06 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850794+0000", 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8433999999999998e-05 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.850811+0000", 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6809000000000001e-05 2024-09-11T09:28:48.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.851646+0000", 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00083492199999999999 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.851695+0000", 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9277000000000003e-05 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:160 1.0 e15/13 1:30306672:devicehealth::main.db.0000000000000000:head v 15'114, mlcod=15'112)", 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:31.853416+0000", 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.489079761999999, 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00093911899999999998, 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853416+0000", 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853416+0000", 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853416+0000", 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.03e-07 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853479+0000", 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2682999999999996e-05 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853488+0000", 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5739999999999996e-06 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853493+0000", 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5520000000000004e-06 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853526+0000", 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3059000000000003e-05 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.853539+0000", 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2310000000000001e-05 2024-09-11T09:28:48.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.854327+0000", 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00078877999999999999 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:31.854355+0000", 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7758000000000001e-05 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.440 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.440 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:48.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:28:48.441 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:48.441+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:48.542 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:48.542+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:48.589 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.589 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:48.589 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:48.589 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:48.598 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:28:48.642 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:48.643+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:48.708 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.708 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:48.708 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:48.708 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:48.708 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:48.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:28:48.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:28:48.759 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:48.759+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:48.844 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:48.845+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:48.867 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.867 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:28:48.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:28:48.867 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:37.466892+0000", 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.401578021000001, 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00087351599999999996, 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466892+0000", 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466892+0000", 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466893+0000", 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9299999999999997e-07 2024-09-11T09:28:48.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466901+0000", 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5040000000000002e-06 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466930+0000", 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8612e-05 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466950+0000", 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9578000000000001e-05 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.466988+0000", 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8699000000000003e-05 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.467016+0000", 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8172999999999999e-05 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.467706+0000", 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00068940799999999997 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:37.467766+0000", 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9849000000000002e-05 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:38.467001+0000", 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.401469494000001, 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00084387799999999995, 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467001+0000", 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467001+0000", 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467001+0000", 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0499999999999999e-07 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467008+0000", 2024-09-11T09:28:48.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467032+0000", 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3397e-05 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467045+0000", 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3482000000000001e-05 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467107+0000", 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2487e-05 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467125+0000", 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7073000000000001e-05 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467806+0000", 2024-09-11T09:28:48.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00068120500000000005 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:38.467845+0000", 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8859999999999997e-05 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4358.0:9 3.1a e24/21 3:5cea7035:::rbd_info:head v 24'2, mlcod=0'0)", 2024-09-11T09:28:48.877 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.703954+0000", 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.16451633800000001, 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066791799999999998, 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.703954+0000", 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.878 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.703954+0000", 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.703954+0000", 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4900000000000002e-07 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.703960+0000", 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6729999999999997e-06 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.703973+0000", 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3638999999999999e-05 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.703982+0000", 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5290000000000008e-06 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.704029+0000", 2024-09-11T09:28:48.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6955e-05 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.704045+0000", 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6317e-05 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.704576+0000", 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00053054600000000001 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:48.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.704622+0000", 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6010000000000002e-05 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:48.882 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:48.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:28:48.944 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:48.945+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:48.991 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:48.991 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:48.991 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:48.991 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:49.001 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:28:49.083 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:49.084+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:49.146 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:49.146+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:49.146 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:49.147 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:49.147 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:49.147 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:49.147 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:49.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:28:49.247 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:49.248+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:49.303 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 12.843639474, 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.304 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:28:49.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.308 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.58211978600000003, 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:28:49.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.310 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:28:49.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.314 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.58087381900000001, 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:49.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:28:49.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:28:49.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:49.319 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:49.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:28:49.364 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:49.365+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:49.435 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:49.435 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:49.435 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:49.435 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:49.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:28:49.449 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:49.450+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:49.550 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:49.551+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:49.587 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:49.588 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:49.588 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:49.588 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:49.588 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:49.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:28:49.680 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:49.681+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:49.702 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:28:49.742 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:49.742 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:28:49.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:28:49.742 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:28:49.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.304279481999998, 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.743 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:28:49.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:28:49.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:28:49.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.910697314, 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053208120000000003, 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7000000000000001e-07 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830585+0000", 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.506e-06 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830589+0000", 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:28:49.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830593+0000", 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9539999999999998e-06 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.834903+0000", 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043093680000000001 2024-09-11T09:28:49.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.834965+0000", 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2754000000000003e-05 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835068+0000", 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102619 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835491+0000", 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042262199999999999 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835845+0000", 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035469000000000001 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835881+0000", 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6059999999999997e-05 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835902+0000", 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1208000000000001e-05 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719809237, 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054255370000000002, 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-07 2024-09-11T09:28:49.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021502+0000", 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2115e-05 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021508+0000", 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021512+0000", 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7439999999999998e-06 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024169+0000", 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002656383 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024216+0000", 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6712000000000003e-05 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024290+0000", 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4051e-05 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025359+0000", 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010691870000000001 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026841+0000", 2024-09-11T09:28:49.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014817669999999999 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026872+0000", 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1517000000000003e-05 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026895+0000", 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.317e-05 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021519+0000", 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719759571000001, 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054331930000000002, 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 110 2024-09-11T09:28:49.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021519+0000", 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021519+0000", 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021520+0000", 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021522+0000", 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2110000000000001e-06 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021526+0000", 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1999999999999996e-06 2024-09-11T09:28:49.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021530+0000", 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4300000000000002e-06 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024775+0000", 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032457330000000002 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024851+0000", 2024-09-11T09:28:49.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5325000000000001e-05 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024979+0000", 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000128804 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026439+0000", 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014597729999999999 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026900+0000", 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046106700000000001 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026923+0000", 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3033000000000001e-05 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026953+0000", 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9280000000000001e-05 2024-09-11T09:28:49.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719745136, 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059776040000000001, 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 111 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021536+0000", 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0449999999999999e-06 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021540+0000", 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.596e-06 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021542+0000", 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6510000000000001e-06 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025459+0000", 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039162329999999999 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025518+0000", 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9490999999999997e-05 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025591+0000", 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2967999999999997e-05 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027405+0000", 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001813989 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027421+0000", 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5807999999999998e-05 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027471+0000", 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0529999999999999e-05 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027511+0000", 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0070000000000001e-05 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719719605000002, 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006066391, 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021560+0000", 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9999999999999999e-07 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021620+0000", 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0795000000000003e-05 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021637+0000", 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6197000000000001e-05 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021643+0000", 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4640000000000003e-06 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025831+0000", 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041879650000000001 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025882+0000", 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0767999999999999e-05 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025956+0000", 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4639000000000001e-05 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027532+0000", 2024-09-11T09:28:49.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001575885 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027563+0000", 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0768000000000001e-05 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027602+0000", 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8522000000000001e-05 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027626+0000", 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4088e-05 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719624874000001, 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061017809999999997, 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021655+0000", 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021659+0000", 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9920000000000001e-06 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021675+0000", 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6217000000000001e-05 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021681+0000", 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026482+0000", 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048014049999999999 2024-09-11T09:28:49.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026534+0000", 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2002000000000001e-05 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026607+0000", 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2936999999999996e-05 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027676+0000", 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010690739999999999 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027687+0000", 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.131e-05 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027734+0000", 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6323999999999999e-05 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027756+0000", 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2310000000000002e-05 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719568291000002, 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071833610000000001, 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-07 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021714+0000", 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1609999999999999e-06 2024-09-11T09:28:49.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021719+0000", 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.04e-06 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021722+0000", 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9129999999999999e-06 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027008+0000", 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052859259999999998 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027057+0000", 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9332000000000001e-05 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027135+0000", 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7935999999999994e-05 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028758+0000", 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016231100000000001 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028793+0000", 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4656000000000002e-05 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028846+0000", 2024-09-11T09:28:49.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2459e-05 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028894+0000", 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8458999999999998e-05 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719549156999999, 2024-09-11T09:28:49.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076141910000000002, 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8099999999999999e-07 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021777+0000", 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7073999999999997e-05 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021782+0000", 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3360000000000002e-06 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021786+0000", 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2289999999999999e-06 2024-09-11T09:28:49.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027795+0000", 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060089870000000004 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027876+0000", 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1749999999999995e-05 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027998+0000", 2024-09-11T09:28:49.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121305 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029211+0000", 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001213638 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029256+0000", 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4236999999999997e-05 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029307+0000", 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0927999999999998e-05 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029344+0000", 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7425999999999997e-05 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.799 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719487789999999, 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083223069999999993, 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:49.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021794+0000", 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021798+0000", 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.551e-06 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021800+0000", 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028416+0000", 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066157569999999999 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028464+0000", 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8229999999999997e-05 2024-09-11T09:28:49.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028537+0000", 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2516999999999997e-05 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029379+0000", 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084212600000000001 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030018+0000", 2024-09-11T09:28:49.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000638909 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030072+0000", 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4292000000000001e-05 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030113+0000", 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1335999999999999e-05 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.719475019000001, 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085277849999999995, 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2700000000000001e-07 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021806+0000", 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.199e-06 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021810+0000", 2024-09-11T09:28:49.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7759999999999999e-06 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021813+0000", 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028937+0000", 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071237260000000004 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028983+0000", 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6090000000000001e-05 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029057+0000", 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4425000000000007e-05 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029591+0000", 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053415799999999996 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030264+0000", 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067266199999999998 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030294+0000", 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9635999999999999e-05 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030332+0000", 2024-09-11T09:28:49.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8195999999999998e-05 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:119 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.030516+0000", 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.710762983999999, 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005472256, 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 119 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030516+0000", 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030516+0000", 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030517+0000", 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3300000000000002e-07 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030551+0000", 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4425999999999999e-05 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030556+0000", 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1569999999999999e-06 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030560+0000", 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031115+0000", 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055544100000000005 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031207+0000", 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1434999999999994e-05 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031280+0000", 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3207000000000006e-05 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032644+0000", 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013646629999999999 2024-09-11T09:28:49.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035918+0000", 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003273934 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035949+0000", 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145999999999999e-05 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035988+0000", 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9733000000000001e-05 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:120 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.030569+0000", 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.710709757, 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054903230000000001, 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 120 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030569+0000", 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030569+0000", 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030570+0000", 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030603+0000", 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3036e-05 2024-09-11T09:28:49.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030608+0000", 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8690000000000003e-06 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030611+0000", 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031546+0000", 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093530999999999996 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031623+0000", 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6234000000000002e-05 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031744+0000", 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121434 2024-09-11T09:28:49.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032683+0000", 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093908100000000003 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035998+0000", 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033145850000000001 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036025+0000", 2024-09-11T09:28:49.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6902000000000001e-05 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036060+0000", 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4986e-05 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:121 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.710661448, 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055023479999999998, 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 121 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030648+0000", 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9677999999999999e-05 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030652+0000", 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3390000000000003e-06 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030655+0000", 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0469999999999998e-06 2024-09-11T09:28:49.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032126+0000", 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001471324 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032200+0000", 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3787000000000004e-05 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032321+0000", 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012095300000000001 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.033140+0000", 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081940300000000002 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036064+0000", 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002923382 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036087+0000", 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2926e-05 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036120+0000", 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3179000000000002e-05 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.316908486999999, 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006285049, 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.831 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424371+0000", 2024-09-11T09:28:49.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5199999999999998e-07 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424412+0000", 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0991000000000001e-05 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424419+0000", 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6469999999999998e-06 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424427+0000", 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7479999999999993e-06 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428503+0000", 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040758069999999999 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428583+0000", 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0288000000000001e-05 2024-09-11T09:28:49.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428703+0000", 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119339 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430520+0000", 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018175999999999999 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430568+0000", 2024-09-11T09:28:49.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7411999999999999e-05 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430618+0000", 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0510000000000003e-05 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430656+0000", 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7455e-05 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.316843064, 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072699360000000003, 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424440+0000", 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5300000000000001e-06 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424450+0000", 2024-09-11T09:28:49.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9499999999999996e-06 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424454+0000", 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429660+0000", 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052059150000000002 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429734+0000", 2024-09-11T09:28:49.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3676000000000004e-05 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429848+0000", 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011422200000000001 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431304+0000", 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001455944 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431655+0000", 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035130099999999999 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431683+0000", 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8172000000000001e-05 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431706+0000", 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2531000000000002e-05 2024-09-11T09:28:49.842 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.316819656, 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0073326670000000002, 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.843 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424460+0000", 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4999999999999999e-07 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424462+0000", 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.942e-06 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424467+0000", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2910000000000001e-06 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424469+0000", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.678e-06 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430155+0000", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056853709999999998 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430201+0000", 2024-09-11T09:28:49.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6239000000000002e-05 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430288+0000", 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6768999999999995e-05 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431736+0000", 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001448048 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431748+0000", 2024-09-11T09:28:49.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1824000000000001e-05 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431773+0000", 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000001e-05 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431792+0000", 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9255000000000001e-05 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.848 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.316805934000001, 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260519999999994, 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-11T09:28:49.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424517+0000", 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3239999999999999e-05 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424523+0000", 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424526+0000", 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.551e-06 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430683+0000", 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061561250000000001 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430725+0000", 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2803999999999999e-05 2024-09-11T09:28:49.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430793+0000", 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7183999999999999e-05 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431886+0000", 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010930270000000001 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432339+0000", 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045296099999999999 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432364+0000", 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5610999999999999e-05 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432399+0000", 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4919999999999998e-05 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.316744234000002, 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081622139999999992, 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:49.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000003e-07 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424540+0000", 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8010000000000003e-06 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424546+0000", 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7679999999999997e-06 2024-09-11T09:28:49.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424551+0000", 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431331+0000", 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067806139999999999 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431371+0000", 2024-09-11T09:28:49.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0110000000000001e-05 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431436+0000", 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5040000000000001e-05 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432172+0000", 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073592299999999998 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432633+0000", 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000460247 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432658+0000", 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5282e-05 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432697+0000", 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9069999999999997e-05 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:11 3.1f e24/21 3:fa654f18:::rbd_id.image_client.0:head v 24'1, mlcod=0'0)", 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.028991789, 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054537040000000002, 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:49.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712294+0000", 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712306+0000", 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712316+0000", 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0239e-05 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712370+0000", 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4259999999999999e-05 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712390+0000", 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0120999999999999e-05 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.717699+0000", 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053085340000000002 2024-09-11T09:28:49.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.717741+0000", 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2079999999999997e-05 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:49.865 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:49.867 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:28:49.867 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:49.867+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:49.967 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:49.968+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:50.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:28:50.010 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:50.010 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:50.010 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:50.010 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:50.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:28:50.103 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:50.104+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:50.138 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:50.138 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:50.155 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:50.156 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:50.161 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:50.161 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:50.162 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:50.162 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:50.162 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:50.169 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:50.169+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:50.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:28:50.172 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:50.172 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:50.189 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:50.189 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:50.206 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:50.206 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:50.222 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:50.223 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:50.269 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:50.270+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:50.313 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:50.313 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:50.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:28:50.406 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:50.407+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:50.465 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:50.465 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:50.471 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:50.472+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:50.473 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:28:50.572 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:50.572+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:50.592 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:28:50.592 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:28:50.593 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-11T09:28:50.593 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-11T09:28:50.601 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:28:50.602 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero size read 2024-09-11T09:28:50.618 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:50.618 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:50.621 INFO:teuthology.orchestra.run.smithi107.stdout:5 write 0x1955e00 thru 0x1958fd5 (0x31d6 bytes) 2024-09-11T09:28:50.622 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xe626a00 2024-09-11T09:28:50.627 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:28:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:6 trunc from 0x1959000 to 0xe626a00 2024-09-11T09:28:50.660 INFO:teuthology.orchestra.run.smithi107.stdout:8 write 0x704c00 thru 0x70d8e7 (0x8ce8 bytes) 2024-09-11T09:28:50.660 INFO:teuthology.orchestra.run.smithi107.stdout:9 read 0x1378400 thru 0x137dbc7 (0x57c8 bytes) 2024-09-11T09:28:50.660 INFO:teuthology.orchestra.run.smithi107.stdout:11 write 0xe15dc00 thru 0xe16d3dd (0xf7de bytes) 2024-09-11T09:28:50.660 INFO:teuthology.orchestra.run.smithi107.stdout:12 punch from 0x32bb600 to 0x32c0000, (0x4a00 bytes) 2024-09-11T09:28:50.660 INFO:teuthology.orchestra.run.smithi107.stdout:19 write 0x8e94000 thru 0x8e9fd6f (0xbd70 bytes) 2024-09-11T09:28:50.660 INFO:teuthology.orchestra.run.smithi107.stdout:20 read 0xa234e00 thru 0xa2387b2 (0x39b3 bytes) 2024-09-11T09:28:50.661 INFO:teuthology.orchestra.run.smithi107.stdout:21 read 0x76fec00 thru 0x770b01c (0xc41d bytes) 2024-09-11T09:28:50.661 INFO:teuthology.orchestra.run.smithi107.stdout:22 write 0xc956200 thru 0xc961ec5 (0xbcc6 bytes) 2024-09-11T09:28:50.661 INFO:teuthology.orchestra.run.smithi107.stdout:24 write 0x3ecb200 thru 0x3ed5c85 (0xaa86 bytes) 2024-09-11T09:28:50.671 INFO:teuthology.orchestra.run.smithi107.stdout:26 trunc from 0xe626a00 to 0x48bae00 2024-09-11T09:28:50.711 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:50.712+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:50.719 INFO:teuthology.orchestra.run.smithi107.stdout:27 write 0x9f15800 thru 0x9f19de2 (0x45e3 bytes) 2024-09-11T09:28:50.738 INFO:teuthology.orchestra.run.smithi107.stdout:28 write 0xd8ef600 thru 0xd8f765b (0x805c bytes) 2024-09-11T09:28:50.744 INFO:teuthology.orchestra.run.smithi107.stdout:31 trunc from 0xd8f7800 to 0xe2ecc00 2024-09-11T09:28:50.757 INFO:teuthology.orchestra.run.smithi107.stdout:34 punch from 0xdf14600 to 0xdf17a00, (0x3400 bytes) 2024-09-11T09:28:50.773 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:50.774+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:50.774 INFO:teuthology.orchestra.run.smithi107.stdout:35 write 0xe410c00 thru 0xe418236 (0x7637 bytes) 2024-09-11T09:28:50.776 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:50.777 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:50.780 INFO:teuthology.orchestra.run.smithi107.stdout:37 read 0x326e200 thru 0x326f1b8 (0xfb9 bytes) 2024-09-11T09:28:50.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:28:50.786 INFO:teuthology.orchestra.run.smithi107.stdout:41 trunc from 0xe418400 to 0x97e8600 2024-09-11T09:28:50.822 INFO:teuthology.orchestra.run.smithi107.stdout:42 write 0xd8f6200 thru 0xd90271e (0xc51f bytes) 2024-09-11T09:28:50.835 INFO:teuthology.orchestra.run.smithi107.stdout:44 trunc from 0xd902800 to 0x3137a00 2024-09-11T09:28:50.856 INFO:teuthology.orchestra.run.smithi107.stdout:45 read 0x1ba9c00 thru 0x1bb4910 (0xad11 bytes) 2024-09-11T09:28:50.858 INFO:teuthology.orchestra.run.smithi107.stdout:46 write 0x1d40800 thru 0x1d43962 (0x3163 bytes) 2024-09-11T09:28:50.858 INFO:teuthology.orchestra.run.smithi107.stdout:47 read 0x2c4c200 thru 0x2c4fae4 (0x38e5 bytes) 2024-09-11T09:28:50.862 INFO:teuthology.orchestra.run.smithi107.stdout:50 punch from 0x2420e00 to 0x2422e00, (0x2000 bytes) 2024-09-11T09:28:50.862 INFO:teuthology.orchestra.run.smithi107.stdout:54 read 0x2e41e00 thru 0x2e48fac (0x71ad bytes) 2024-09-11T09:28:50.862 INFO:teuthology.orchestra.run.smithi107.stdout:56 punch from 0x4a5e00 to 0x4ab200, (0x5400 bytes) 2024-09-11T09:28:50.872 INFO:teuthology.orchestra.run.smithi107.stdout:57 write 0x5dd6e00 thru 0x5ddf37a (0x857b bytes) 2024-09-11T09:28:50.874 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:50.875+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:50.876 INFO:teuthology.orchestra.run.smithi107.stdout:59 read 0x14d2800 thru 0x14e0eed (0xe6ee bytes) 2024-09-11T09:28:50.876 INFO:teuthology.orchestra.run.smithi107.stdout:60 trunc from 0x5ddf400 to 0x2f11200 2024-09-11T09:28:50.893 INFO:teuthology.orchestra.run.smithi107.stdout:62 read 0x15ccc00 thru 0x15dacf3 (0xe0f4 bytes) 2024-09-11T09:28:50.907 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:50.907 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:50.916 INFO:teuthology.orchestra.run.smithi107.stdout:65 write 0xbc07000 thru 0xbc0b6f5 (0x46f6 bytes) 2024-09-11T09:28:50.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:28:50.918 INFO:teuthology.orchestra.run.smithi107.stdout:67 read 0x2742a00 thru 0x2750766 (0xdd67 bytes) 2024-09-11T09:28:50.918 INFO:teuthology.orchestra.run.smithi107.stdout:71 write 0x307b400 thru 0x308a7c3 (0xf3c4 bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:73 punch from 0xa476600 to 0xa486200, (0xfc00 bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:75 punch from 0x8b45a00 to 0x8b48000, (0x2600 bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:76 punch from 0x8082a00 to 0x8091e00, (0xf400 bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:77 read 0x3432e00 thru 0x34368cc (0x3acd bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:78 write 0x6179a00 thru 0x61899a4 (0xffa5 bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:79 punch from 0x8d91a00 to 0x8d9c800, (0xae00 bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:83 write 0x32eae00 thru 0x32ee2ab (0x34ac bytes) 2024-09-11T09:28:50.921 INFO:teuthology.orchestra.run.smithi107.stdout:85 read 0xaff2e00 thru 0xb002a8b (0xfc8c bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:88 read 0x2a6e00 thru 0x2af8cb (0x8acc bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:89 read 0x63b8600 thru 0x63c401d (0xba1e bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:90 punch from 0xac71800 to 0xac7ac00, (0x9400 bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:92 write 0x2e7aa00 thru 0x2e7f5fb (0x4bfc bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:94 read 0x3142600 thru 0x31506fa (0xe0fb bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:95 read 0xb11b800 thru 0xb126ea7 (0xb6a8 bytes) 2024-09-11T09:28:50.922 INFO:teuthology.orchestra.run.smithi107.stdout:96 read 0xa8ab800 thru 0xa8af41b (0x3c1c bytes) 2024-09-11T09:28:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:97 write 0x1043000 thru 0x104b28c (0x828d bytes) 2024-09-11T09:28:50.923 INFO:teuthology.orchestra.run.smithi107.stdout:98 clone 1 order 24 su 524288 sc 7 2024-09-11T09:28:50.934 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-11T09:28:50.935+0000 7fef02752640 -1 mon.a@0(leader).osd e25 definitely_dead 0 2024-09-11T09:28:50.975 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:50.975+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:51.041 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.041 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:28:51.096 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:51.097+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:51.159 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.159 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.167 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:28:51.177 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:51.178+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:51.278 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:51.278+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:51.315 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.315 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.325 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:28:51.379 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:51.379+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:51.432 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.432 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:28:51.480 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:51.480+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:51.581 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:51.581+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:51.588 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.588 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.596 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:28:51.682 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:51.683+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:51.705 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.705 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.710 INFO:teuthology.orchestra.run.smithi017.stderr:marked out osd.2. 2024-09-11T09:28:51.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:28:51.719 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:28:51.783 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:51.784+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:51.876 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:51.877 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:51.884 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:51.884+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:51.886 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:28:51.985 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:51.986+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:52.033 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:52.033 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:52.042 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:28:52.086 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:52.087+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:52.156 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:52.157 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:52.166 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:28:52.187 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:52.188+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:52.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:28:52.288 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:52.288+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:52.310 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:52.310 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:52.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:28:52.357 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:52.357 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:52.374 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:52.374 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:52.388 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:52.389+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:52.390 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:52.390 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:52.407 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:52.407 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:52.424 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:52.424 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:52.441 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:52.441 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:52.466 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:28:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:28:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:28:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.466 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.028149386999999, 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:28:52.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:28:52.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.634567219000001, 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053208120000000003, 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 37 2024-09-11T09:28:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830582+0000", 2024-09-11T09:28:52.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7000000000000001e-07 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830585+0000", 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.506e-06 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830589+0000", 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.830593+0000", 2024-09-11T09:28:52.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9539999999999998e-06 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.834903+0000", 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043093680000000001 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.834965+0000", 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2754000000000003e-05 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835068+0000", 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000102619 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835491+0000", 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042262199999999999 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835845+0000", 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035469000000000001 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835881+0000", 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6059999999999997e-05 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.835902+0000", 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1208000000000001e-05 2024-09-11T09:28:52.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443679142000001, 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054255370000000002, 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 109 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021470+0000", 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-07 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021502+0000", 2024-09-11T09:28:52.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2115e-05 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021508+0000", 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.485e-06 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021512+0000", 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7439999999999998e-06 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024169+0000", 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002656383 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024216+0000", 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6712000000000003e-05 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024290+0000", 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4051e-05 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025359+0000", 2024-09-11T09:28:52.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010691870000000001 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026841+0000", 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014817669999999999 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026872+0000", 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1517000000000003e-05 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026895+0000", 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.317e-05 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021519+0000", 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443629476000002, 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054331930000000002, 2024-09-11T09:28:52.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 110 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021519+0000", 2024-09-11T09:28:52.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021519+0000", 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021520+0000", 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021522+0000", 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2110000000000001e-06 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021526+0000", 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1999999999999996e-06 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021530+0000", 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4300000000000002e-06 2024-09-11T09:28:52.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024775+0000", 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032457330000000002 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024851+0000", 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5325000000000001e-05 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.024979+0000", 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000128804 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026439+0000", 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014597729999999999 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026900+0000", 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00046106700000000001 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026923+0000", 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3033000000000001e-05 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026953+0000", 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9280000000000001e-05 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443615041000001, 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059776040000000001, 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 111 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021536+0000", 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0449999999999999e-06 2024-09-11T09:28:52.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021540+0000", 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.596e-06 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021542+0000", 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6510000000000001e-06 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025459+0000", 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039162329999999999 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025518+0000", 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9490999999999997e-05 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025591+0000", 2024-09-11T09:28:52.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2967999999999997e-05 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027405+0000", 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001813989 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027421+0000", 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5807999999999998e-05 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027471+0000", 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0529999999999999e-05 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027511+0000", 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0070000000000001e-05 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.503 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443589509999999, 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006066391, 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021560+0000", 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9999999999999999e-07 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021620+0000", 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0795000000000003e-05 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021637+0000", 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6197000000000001e-05 2024-09-11T09:28:52.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021643+0000", 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4640000000000003e-06 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025831+0000", 2024-09-11T09:28:52.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041879650000000001 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025882+0000", 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0767999999999999e-05 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025956+0000", 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4639000000000001e-05 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027532+0000", 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001575885 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027563+0000", 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0768000000000001e-05 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027602+0000", 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8522000000000001e-05 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027626+0000", 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4088e-05 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:52.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443494779000002, 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061017809999999997, 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021655+0000", 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021659+0000", 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9920000000000001e-06 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021675+0000", 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6217000000000001e-05 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021681+0000", 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2024-09-11T09:28:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026482+0000", 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048014049999999999 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026534+0000", 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2002000000000001e-05 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026607+0000", 2024-09-11T09:28:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2936999999999996e-05 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027676+0000", 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010690739999999999 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027687+0000", 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.131e-05 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027734+0000", 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6323999999999999e-05 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027756+0000", 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2310000000000002e-05 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443438195999999, 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071833610000000001, 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:52.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-07 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021714+0000", 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1609999999999999e-06 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021719+0000", 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.04e-06 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021722+0000", 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9129999999999999e-06 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027008+0000", 2024-09-11T09:28:52.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052859259999999998 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027057+0000", 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9332000000000001e-05 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027135+0000", 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7935999999999994e-05 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028758+0000", 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016231100000000001 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028793+0000", 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4656000000000002e-05 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028846+0000", 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2459e-05 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028894+0000", 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8458999999999998e-05 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443419062, 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076141910000000002, 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8099999999999999e-07 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021777+0000", 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7073999999999997e-05 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021782+0000", 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3360000000000002e-06 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021786+0000", 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2289999999999999e-06 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027795+0000", 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060089870000000004 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027876+0000", 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1749999999999995e-05 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027998+0000", 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121305 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029211+0000", 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001213638 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029256+0000", 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4236999999999997e-05 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029307+0000", 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0927999999999998e-05 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029344+0000", 2024-09-11T09:28:52.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7425999999999997e-05 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443357695, 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083223069999999993, 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021794+0000", 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021798+0000", 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.551e-06 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021800+0000", 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028416+0000", 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066157569999999999 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028464+0000", 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8229999999999997e-05 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028537+0000", 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2516999999999997e-05 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029379+0000", 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084212600000000001 2024-09-11T09:28:52.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030018+0000", 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000638909 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030072+0000", 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4292000000000001e-05 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030113+0000", 2024-09-11T09:28:52.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1335999999999999e-05 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.443344924000002, 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085277849999999995, 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2700000000000001e-07 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021806+0000", 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.199e-06 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021810+0000", 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7759999999999999e-06 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021813+0000", 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028937+0000", 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071237260000000004 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028983+0000", 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6090000000000001e-05 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029057+0000", 2024-09-11T09:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4425000000000007e-05 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029591+0000", 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053415799999999996 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030264+0000", 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067266199999999998 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030294+0000", 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9635999999999999e-05 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030332+0000", 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8195999999999998e-05 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:119 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.030516+0000", 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.434632889, 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005472256, 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 119 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030516+0000", 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030516+0000", 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030517+0000", 2024-09-11T09:28:52.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3300000000000002e-07 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030551+0000", 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4425999999999999e-05 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030556+0000", 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1569999999999999e-06 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030560+0000", 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4809999999999998e-06 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031115+0000", 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00055544100000000005 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031207+0000", 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1434999999999994e-05 2024-09-11T09:28:52.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031280+0000", 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3207000000000006e-05 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032644+0000", 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013646629999999999 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035918+0000", 2024-09-11T09:28:52.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003273934 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035949+0000", 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0145999999999999e-05 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035988+0000", 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9733000000000001e-05 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:120 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.030569+0000", 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.434579662000001, 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054903230000000001, 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 120 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030569+0000", 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030569+0000", 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030570+0000", 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030603+0000", 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3036e-05 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030608+0000", 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8690000000000003e-06 2024-09-11T09:28:52.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030611+0000", 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2030000000000001e-06 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031546+0000", 2024-09-11T09:28:52.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093530999999999996 2024-09-11T09:28:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031623+0000", 2024-09-11T09:28:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6234000000000002e-05 2024-09-11T09:28:52.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.031744+0000", 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121434 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032683+0000", 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00093908100000000003 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.035998+0000", 2024-09-11T09:28:52.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033145850000000001 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036025+0000", 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6902000000000001e-05 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036060+0000", 2024-09-11T09:28:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4986e-05 2024-09-11T09:28:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:121 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.434531353000001, 2024-09-11T09:28:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055023479999999998, 2024-09-11T09:28:52.559 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 121 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030618+0000", 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030648+0000", 2024-09-11T09:28:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9677999999999999e-05 2024-09-11T09:28:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030652+0000", 2024-09-11T09:28:52.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3390000000000003e-06 2024-09-11T09:28:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030655+0000", 2024-09-11T09:28:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0469999999999998e-06 2024-09-11T09:28:52.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032126+0000", 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001471324 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032200+0000", 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3787000000000004e-05 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.032321+0000", 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012095300000000001 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.033140+0000", 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00081940300000000002 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036064+0000", 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002923382 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036087+0000", 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2926e-05 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.036120+0000", 2024-09-11T09:28:52.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3179000000000002e-05 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.040778392, 2024-09-11T09:28:52.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006285049, 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424371+0000", 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5199999999999998e-07 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424412+0000", 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0991000000000001e-05 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424419+0000", 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6469999999999998e-06 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424427+0000", 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7479999999999993e-06 2024-09-11T09:28:52.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428503+0000", 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040758069999999999 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428583+0000", 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0288000000000001e-05 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428703+0000", 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119339 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430520+0000", 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018175999999999999 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430568+0000", 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7411999999999999e-05 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430618+0000", 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0510000000000003e-05 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430656+0000", 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7455e-05 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.575 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.040712969000001, 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072699360000000003, 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.576 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424440+0000", 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5300000000000001e-06 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424450+0000", 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9499999999999996e-06 2024-09-11T09:28:52.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424454+0000", 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429660+0000", 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052059150000000002 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429734+0000", 2024-09-11T09:28:52.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3676000000000004e-05 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429848+0000", 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011422200000000001 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431304+0000", 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001455944 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431655+0000", 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035130099999999999 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431683+0000", 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8172000000000001e-05 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431706+0000", 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2531000000000002e-05 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.040689561000001, 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0073326670000000002, 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424460+0000", 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4999999999999999e-07 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424462+0000", 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.942e-06 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424467+0000", 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2910000000000001e-06 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424469+0000", 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.678e-06 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430155+0000", 2024-09-11T09:28:52.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056853709999999998 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430201+0000", 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6239000000000002e-05 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430288+0000", 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6768999999999995e-05 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431736+0000", 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001448048 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431748+0000", 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1824000000000001e-05 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431773+0000", 2024-09-11T09:28:52.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000001e-05 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431792+0000", 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9255000000000001e-05 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:52.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.040675838999999, 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260519999999994, 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424517+0000", 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3239999999999999e-05 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424523+0000", 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424526+0000", 2024-09-11T09:28:52.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.551e-06 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430683+0000", 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061561250000000001 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430725+0000", 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2803999999999999e-05 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430793+0000", 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7183999999999999e-05 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431886+0000", 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010930270000000001 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432339+0000", 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045296099999999999 2024-09-11T09:28:52.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432364+0000", 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5610999999999999e-05 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432399+0000", 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4919999999999998e-05 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 21.040614138999999, 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081622139999999992, 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000003e-07 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424540+0000", 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8010000000000003e-06 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424546+0000", 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7679999999999997e-06 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424551+0000", 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431331+0000", 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067806139999999999 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431371+0000", 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0110000000000001e-05 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431436+0000", 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5040000000000001e-05 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432172+0000", 2024-09-11T09:28:52.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073592299999999998 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432633+0000", 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000460247 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432658+0000", 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5282e-05 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432697+0000", 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9069999999999997e-05 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:11 3.1f e24/21 3:fa654f18:::rbd_id.image_client.0:head v 24'1, mlcod=0'0)", 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7528616939999999, 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054537040000000002, 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:52.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712287+0000", 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7300000000000002e-07 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712294+0000", 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0859999999999998e-06 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712306+0000", 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2111e-05 2024-09-11T09:28:52.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712316+0000", 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0239e-05 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712370+0000", 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4259999999999999e-05 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.712390+0000", 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0120999999999999e-05 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.717699+0000", 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0053085340000000002 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.717741+0000", 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2079999999999997e-05 2024-09-11T09:28:52.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:52.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:52.607 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:52.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:28:52.609 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:52.609+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:52.709 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:52.709+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:52.712 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:28:52.759 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:52.760 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:52.760 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:52.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:28:52.810 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:52.811+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:52.911 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:52.911+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:52.911 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:52.911 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:52.912 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:52.912 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:52.912 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:52.920 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:28:53.012 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:53.012+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:53.026 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.699189+0000", 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3260455220000003, 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055671330000000002, 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 9 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699189+0000", 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699189+0000", 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699192+0000", 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3800000000000001e-06 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699214+0000", 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.211e-05 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699229+0000", 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5480000000000001e-05 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699239+0000", 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9969999999999993e-06 2024-09-11T09:28:53.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699342+0000", 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010292499999999999 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.699349+0000", 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1049999999999997e-06 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.703504+0000", 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041542669999999997 2024-09-11T09:28:53.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.703575+0000", 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1798999999999995e-05 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.703691+0000", 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000115561 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.704612+0000", 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00092130200000000004 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.704683+0000", 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0499000000000004e-05 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.704727+0000", 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4613000000000001e-05 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.704756+0000", 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9094999999999998e-05 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:10 3.1c e24/21 3:3831950c:::rbd_directory:head v 24'1, mlcod=0'0)", 2024-09-11T09:28:53.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.708438+0000", 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3167971300000003, 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00085003399999999995, 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708438+0000", 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708438+0000", 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708438+0000", 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1199999999999999e-07 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708442+0000", 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1899999999999997e-06 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708453+0000", 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0991e-05 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708461+0000", 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9899999999999997e-06 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708519+0000", 2024-09-11T09:28:53.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8121999999999997e-05 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708543+0000", 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3394999999999999e-05 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.709238+0000", 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000695215 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.709288+0000", 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9919000000000001e-05 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:12 3.1 e24/21 3:846053a3:::rbd_header.11065aee08ca:head v 24'1, mlcod=0'0)", 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.718879+0000", 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.3063557560000003, 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001555297, 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718879+0000", 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718879+0000", 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718879+0000", 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8699999999999999e-07 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718883+0000", 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4079999999999998e-06 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718891+0000", 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7790000000000001e-06 2024-09-11T09:28:53.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718900+0000", 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.405e-06 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718942+0000", 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2120999999999999e-05 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.718955+0000", 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3478e-05 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.720389+0000", 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014332520000000001 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.720434+0000", 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5667000000000002e-05 2024-09-11T09:28:53.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:25 3.1 e24/21 3:846053a3:::rbd_header.11065aee08ca:head v 24'2, mlcod=0'0)", 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.733982+0000", 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.2912526309999999, 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056546299999999999, 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.733982+0000", 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.733982+0000", 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.733983+0000", 2024-09-11T09:28:53.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2500000000000004e-07 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.733989+0000", 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0430000000000004e-06 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.734000+0000", 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1304999999999999e-05 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.734006+0000", 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6960000000000002e-06 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.734058+0000", 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2111999999999998e-05 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.734075+0000", 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6665999999999999e-05 2024-09-11T09:28:53.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.734505+0000", 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00043043399999999999 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.734548+0000", 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2382000000000002e-05 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4385.0:12 3.1 e24/21 3:846053a3:::rbd_header.11065aee08ca:head v 24'3, mlcod=24'1)", 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.891486+0000", 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1337489649999997, 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00063673500000000004, 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891486+0000", 2024-09-11T09:28:53.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891486+0000", 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891487+0000", 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891495+0000", 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3960000000000006e-06 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891504+0000", 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7029999999999995e-06 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891509+0000", 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9740000000000001e-06 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891562+0000", 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3174999999999998e-05 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.891585+0000", 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3034999999999999e-05 2024-09-11T09:28:53.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.892079+0000", 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049440599999999997 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.892123+0000", 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3299e-05 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4385.0:433 3.1a 3:598a475f:::rbd_object_map.11065aee08ca.0000000000000003:head [writefull 0~46] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.716653+0000", 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.30858153599999999, 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048876120000000004, 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4385", 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/2255862569", 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 433 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716653+0000", 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716653+0000", 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716655+0000", 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5659999999999999e-06 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716661+0000", 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3729999999999998e-06 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716672+0000", 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0688e-05 2024-09-11T09:28:53.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716688+0000", 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5688999999999998e-05 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716739+0000", 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1638000000000003e-05 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.716895+0000", 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015551000000000001 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.717032+0000", 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013755700000000001 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.719186+0000", 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021537269999999998 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.721362+0000", 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002176084 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.721494+0000", 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013215199999999999 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.721541+0000", 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6628000000000001e-05 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPush(3.0 27/26 [PushOp(3:02638d11:::rbd_data.11065aee08ca.000000000000002b:head, version: 25'1, data_included: [464896~37888], data_size: 37888, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:02638d11:::rbd_data.11065aee08ca.000000000000002b:head@25'1, size: 502784, copy_subset: [(0, 502784)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 502784, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.720820+0000", 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.304415194, 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059163800000000002, 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.720820+0000", 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.720820+0000", 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.720820+0000", 2024-09-11T09:28:53.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4499999999999997e-07 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.720909+0000", 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8993999999999995e-05 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.720928+0000", 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8870999999999999e-05 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.720952+0000", 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3592e-05 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.721022+0000", 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0018000000000006e-05 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.721029+0000", 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4460000000000004e-06 2024-09-11T09:28:53.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.721411+0000", 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038227200000000002 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:251 3.1 e27/21 3:846053a3:::rbd_header.11065aee08ca:head v 27'44, mlcod=25'42)", 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.732102+0000", 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.29313300599999997, 2024-09-11T09:28:53.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00062891900000000003, 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732102+0000", 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732102+0000", 2024-09-11T09:28:53.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732102+0000", 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732110+0000", 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8790000000000003e-06 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732120+0000", 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.01e-05 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732127+0000", 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2740000000000002e-06 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732174+0000", 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6118000000000001e-05 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732203+0000", 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9329e-05 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732708+0000", 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00050492699999999996 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.732731+0000", 2024-09-11T09:28:53.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2921999999999998e-05 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:271 3.1c e27/21 3:3831950c:::rbd_directory:head v 27'3, mlcod=24'1)", 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.746957+0000", 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.27827745500000001, 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057146900000000001, 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.746957+0000", 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.746957+0000", 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.746958+0000", 2024-09-11T09:28:53.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4199999999999995e-07 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.746964+0000", 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5260000000000001e-06 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.746973+0000", 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000005e-06 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.746978+0000", 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4269999999999999e-06 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.747034+0000", 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5327000000000001e-05 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.747056+0000", 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2911000000000001e-05 2024-09-11T09:28:53.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.747483+0000", 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042659700000000002 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.747529+0000", 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5788999999999998e-05 2024-09-11T09:28:53.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:273 3.9 3:900f5e89:::rbd_header.11064c3f4ab3:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.750434+0000", 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.27480084300000002, 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023296409999999999, 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 273 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750434+0000", 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750434+0000", 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750435+0000", 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.54e-07 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750440+0000", 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750450+0000", 2024-09-11T09:28:53.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6679999999999996e-06 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750455+0000", 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750500+0000", 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5002e-05 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750579+0000", 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8763999999999993e-05 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750710+0000", 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013152899999999999 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752521+0000", 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001811202 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752696+0000", 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000175038 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752730+0000", 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3525000000000002e-05 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752764+0000", 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3710000000000001e-05 2024-09-11T09:28:53.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:274 3.9 3:900f5e89:::rbd_header.11064c3f4ab3:head [call rbd.set_stripe_unit_count in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.752976+0000", 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.27225919500000001, 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024869240000000002, 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 274 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752976+0000", 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752976+0000", 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752976+0000", 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1799999999999995e-07 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752980+0000", 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752989+0000", 2024-09-11T09:28:53.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7450000000000001e-06 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752994+0000", 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4319999999999998e-06 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753035+0000", 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1152999999999997e-05 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753111+0000", 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5704000000000006e-05 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753334+0000", 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022288399999999999 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753774+0000", 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000440505 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.755397+0000", 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001622276 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.755427+0000", 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0012e-05 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.755463+0000", 2024-09-11T09:28:53.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5969000000000001e-05 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:288 3.9 3:900f5e89:::rbd_header.11064c3f4ab3:head [watch watch cookie 140373288944432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.767092+0000", 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.25814272300000002, 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011839870000000001, 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 288 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767092+0000", 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767092+0000", 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767093+0000", 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8999999999999996e-07 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767095+0000", 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4569999999999999e-06 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767105+0000", 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6039999999999999e-06 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767108+0000", 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7230000000000002e-06 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767141+0000", 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2820000000000001e-05 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767191+0000", 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9310000000000001e-05 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767292+0000", 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101167 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.767822+0000", 2024-09-11T09:28:53.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053071399999999995 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768152+0000", 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00032906800000000001 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768206+0000", 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4215999999999997e-05 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768276+0000", 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0318e-05 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:289 3.9 3:900f5e89:::rbd_header.11064c3f4ab3:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.768497+0000", 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.25673787300000001, 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0024823720000000001, 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 289 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768497+0000", 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768497+0000", 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768498+0000", 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999997e-07 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768502+0000", 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4859999999999999e-06 2024-09-11T09:28:53.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768512+0000", 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4369999999999996e-06 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768517+0000", 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8720000000000001e-06 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768557+0000", 2024-09-11T09:28:53.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0809000000000002e-05 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768634+0000", 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7069000000000004e-05 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768870+0000", 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023541600000000001 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.769298+0000", 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000428548 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.770891+0000", 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0015923420000000001 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.770942+0000", 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.770979+0000", 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6953999999999999e-05 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:290 3.a e27/21 3:535385f0:::rbd_children:head v 27'1, mlcod=0'0)", 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.771805+0000", 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.253430302, 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00057335999999999997, 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.098 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771805+0000", 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771805+0000", 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771805+0000", 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771808+0000", 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.349e-06 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771817+0000", 2024-09-11T09:28:53.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1500000000000005e-06 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771824+0000", 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5769999999999999e-06 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771869+0000", 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5303000000000003e-05 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.771885+0000", 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6025e-05 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.772340+0000", 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045451900000000002 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.772378+0000", 2024-09-11T09:28:53.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8151000000000001e-05 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:298 3.9 3:900f5e89:::rbd_header.11064c3f4ab3:head [watch unwatch cookie 140373288944432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.777359+0000", 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.24787542700000001, 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012796800000000001, 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 298 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777359+0000", 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777359+0000", 2024-09-11T09:28:53.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777361+0000", 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9870000000000002e-06 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777371+0000", 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2129999999999997e-06 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777397+0000", 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6803000000000001e-05 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777411+0000", 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3332999999999999e-05 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777531+0000", 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000120568 2024-09-11T09:28:53.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777664+0000", 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013236000000000001 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.777838+0000", 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017403399999999999 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.778227+0000", 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038935199999999998 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.778540+0000", 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00031317500000000001 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.778596+0000", 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5510000000000002e-05 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.778639+0000", 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3344999999999999e-05 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:302 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.781674+0000", 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.24356049900000001, 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054239499999999999, 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 302 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781674+0000", 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781674+0000", 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781676+0000", 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0329999999999998e-06 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781686+0000", 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.004e-05 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781712+0000", 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5429e-05 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781725+0000", 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3377000000000001e-05 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.781849+0000", 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012376999999999999 2024-09-11T09:28:53.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.782076+0000", 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00022654899999999999 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.782217+0000", 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000141197 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4385.0:466 3.1 e27/21 3:846053a3:::rbd_header.11065aee08ca:head v 27'45, mlcod=26'43)", 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.933244+0000", 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.091990636000000001, 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00066476000000000003, 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933244+0000", 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933244+0000", 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933245+0000", 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1400000000000001e-06 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933253+0000", 2024-09-11T09:28:53.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1219999999999995e-06 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933266+0000", 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2869e-05 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933273+0000", 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.229e-06 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933337+0000", 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4920999999999997e-05 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933362+0000", 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4167000000000001e-05 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933857+0000", 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000494904 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.933909+0000", 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2407999999999997e-05 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4385.0:468 3.1 e27/21 3:846053a3:::rbd_header.11065aee08ca:head v 27'46, mlcod=27'44)", 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.934681+0000", 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.090553433000000003, 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0019590020000000001, 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934681+0000", 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934681+0000", 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934682+0000", 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3799999999999997e-07 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934688+0000", 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5660000000000003e-06 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934697+0000", 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0839999999999992e-06 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934703+0000", 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8229999999999997e-06 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934746+0000", 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2709000000000001e-05 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.934767+0000", 2024-09-11T09:28:53.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1401000000000001e-05 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.936593+0000", 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001825508 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.936640+0000", 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7373000000000002e-05 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:306 3.1 e27/21 3:846053a3:::rbd_header.11065aee08ca:head v 27'47, mlcod=27'45)", 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.946035+0000", 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.079200038, 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00059061400000000005, 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946035+0000", 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946035+0000", 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946036+0000", 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.282e-06 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946047+0000", 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0674e-05 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946062+0000", 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5671999999999998e-05 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946069+0000", 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3609999999999997e-06 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946118+0000", 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9490999999999998e-05 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946136+0000", 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7450999999999999e-05 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946577+0000", 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00044112699999999998 2024-09-11T09:28:53.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.946625+0000", 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8556000000000001e-05 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:331 3.9 3:900f5e89:::rbd_header.11064c3f4ab3:head [watch watch cookie 140373288926400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.963648+0000", 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.061586650999999999, 2024-09-11T09:28:53.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001258704, 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 331 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963648+0000", 2024-09-11T09:28:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963648+0000", 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963649+0000", 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4799999999999997e-07 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963653+0000", 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8160000000000004e-06 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963664+0000", 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0725e-05 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963669+0000", 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0089999999999998e-06 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963709+0000", 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0565000000000002e-05 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963783+0000", 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3891000000000001e-05 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 5", 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.963955+0000", 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017154 2024-09-11T09:28:53.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.964501+0000", 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00054611099999999995 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.964798+0000", 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00029771600000000001 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.964848+0000", 2024-09-11T09:28:53.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0006000000000002e-05 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.964907+0000", 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8477000000000003e-05 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.134 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:53.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:99 trunc from 0xbc0b800 to 0xc537600 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:100 trunc from 0xc537600 to 0x65d3400 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:103 write 0x1052800 thru 0x106070f (0xdf10 bytes) 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:105 punch from 0x5968a00 to 0x596da00, (0x5000 bytes) 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:107 punch from 0xb2a000 to 0xb2ec00, (0x4c00 bytes) 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:108 write 0x4e6fc00 thru 0x4e791ab (0x95ac bytes) 2024-09-11T09:28:53.137 INFO:teuthology.orchestra.run.smithi107.stdout:111 read 0x5945e00 thru 0x5948e49 (0x304a bytes) 2024-09-11T09:28:53.138 INFO:teuthology.orchestra.run.smithi107.stdout:112 clone 2 order 21 su 32768 sc 8 2024-09-11T09:28:53.138 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:53.136+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:53.236 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:53.236+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:53.240 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:53.240 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:53.240 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:53.249 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:28:53.337 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:53.338+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:53.392 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:53.392 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:53.392 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:53.392 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:53.392 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:53.400 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:28:53.443 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:53.444+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:53.508 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:53.508 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:28:53.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:28:53.508 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:28:53.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4208.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:36.459620+0000", 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.045232013, 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0014866669999999999, 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4208", 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.17:0/1682482807", 2024-09-11T09:28:53.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459620+0000", 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459620+0000", 2024-09-11T09:28:53.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459624+0000", 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.092e-06 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459631+0000", 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5519999999999999e-06 2024-09-11T09:28:53.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459641+0000", 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0176e-05 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459651+0000", 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0037999999999999e-05 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459705+0000", 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3090000000000002e-05 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.459878+0000", 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017356000000000001 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 2", 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.460063+0000", 2024-09-11T09:28:53.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000185236 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.460755+0000", 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000691276 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.461013+0000", 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025803499999999999 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.461064+0000", 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0948000000000001e-05 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:36.461106+0000", 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2663999999999997e-05 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4208.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:37.461153+0000", 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.043699014000001, 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067391650000000001, 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4208", 2024-09-11T09:28:53.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.17:0/1682482807", 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461153+0000", 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461153+0000", 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461155+0000", 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2369999999999999e-06 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461168+0000", 2024-09-11T09:28:53.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3335e-05 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461186+0000", 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8110000000000001e-05 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461199+0000", 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2711000000000001e-05 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461275+0000", 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5814000000000003e-05 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.461280+0000", 2024-09-11T09:28:53.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.541e-06 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.466271+0000", 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0049904349999999997 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.466372+0000", 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010101400000000001 2024-09-11T09:28:53.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.466478+0000", 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106339 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.467237+0000", 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075928900000000001 2024-09-11T09:28:53.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.467802+0000", 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056480699999999996 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.467838+0000", 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5936999999999998e-05 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:37.467892+0000", 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3596e-05 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4208.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:38.462876+0000", 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.041975756999999, 2024-09-11T09:28:53.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0050932599999999996, 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4208", 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.17:0/1682482807", 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 7 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462876+0000", 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462876+0000", 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462878+0000", 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0679999999999999e-06 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462886+0000", 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2989999999999997e-06 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462906+0000", 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9417000000000001e-05 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462914+0000", 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6090000000000001e-06 2024-09-11T09:28:53.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.462998+0000", 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3426999999999999e-05 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.463003+0000", 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2009999999999998e-06 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.466562+0000", 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035589990000000002 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.466628+0000", 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6044000000000007e-05 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.466738+0000", 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110229 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.467599+0000", 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00086070999999999999 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.467899+0000", 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000300501 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.467936+0000", 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7144999999999998e-05 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:38.467969+0000", 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2611000000000003e-05 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:48.705800+0000", 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.7990520310000004, 2024-09-11T09:28:53.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.003780186, 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 10 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705800+0000", 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705800+0000", 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705801+0000", 2024-09-11T09:28:53.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6899999999999999e-06 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705807+0000", 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6149999999999996e-06 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705816+0000", 2024-09-11T09:28:53.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5149999999999995e-06 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705825+0000", 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.259e-06 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705895+0000", 2024-09-11T09:28:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0080000000000007e-05 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op must wait for map", 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.705901+0000", 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0399999999999998e-06 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.707717+0000", 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001816025 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.707889+0000", 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172103 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:53.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.708085+0000", 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019605999999999999 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.709152+0000", 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010669049999999999 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.709494+0000", 2024-09-11T09:28:53.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034251699999999997 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.709536+0000", 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1176e-05 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:48.709580+0000", 2024-09-11T09:28:53.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4200999999999999e-05 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:273 3.9 e27/26 3:900f5e89:::rbd_header.11064c3f4ab3:head v 27'8, mlcod=25'6)", 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.750778+0000", 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.75407363000000005, 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018182319999999999, 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750778+0000", 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750778+0000", 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750779+0000", 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750785+0000", 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.781e-06 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750799+0000", 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3371e-05 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750806+0000", 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6569999999999997e-06 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750859+0000", 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2664999999999998e-05 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.750889+0000", 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9478000000000001e-05 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752558+0000", 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001669467 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.752596+0000", 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8126000000000001e-05 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.547 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:274 3.9 e27/26 3:900f5e89:::rbd_header.11064c3f4ab3:head v 27'9, mlcod=25'7)", 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.753423+0000", 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.75142904099999996, 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018782219999999999, 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753423+0000", 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753423+0000", 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753423+0000", 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4899999999999995e-07 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753428+0000", 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753441+0000", 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2577e-05 2024-09-11T09:28:53.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753449+0000", 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8609999999999998e-06 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753493+0000", 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4552000000000003e-05 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.753510+0000", 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7133e-05 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.755267+0000", 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017561079999999999 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.755301+0000", 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4211999999999997e-05 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4358.0:289 3.9 e27/26 3:900f5e89:::rbd_header.11064c3f4ab3:head v 27'11, mlcod=27'9)", 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:52.768929+0000", 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.73592259599999998, 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018429480000000001, 2024-09-11T09:28:53.553 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768929+0000", 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768929+0000", 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768929+0000", 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768934+0000", 2024-09-11T09:28:53.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7659999999999998e-06 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768944+0000", 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4089999999999992e-06 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768948+0000", 2024-09-11T09:28:53.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2910000000000001e-06 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.768992+0000", 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.401e-05 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.769012+0000", 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9570999999999999e-05 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.770742+0000", 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0017301180000000001 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:52.770772+0000", 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0289000000000001e-05 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4483.0:129 3.5 3:a50d5ea9:::rbd_data.11065aee08ca.000000000000000b:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.071558+0000", 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.43329320500000001, 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001324913, 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4483", 2024-09-11T09:28:53.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 129 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071558+0000", 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071558+0000", 2024-09-11T09:28:53.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071559+0000", 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071561+0000", 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8369999999999999e-06 2024-09-11T09:28:53.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071566+0000", 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8309999999999997e-06 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071572+0000", 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2970000000000003e-06 2024-09-11T09:28:53.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.072722+0000", 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001149759 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.072789+0000", 2024-09-11T09:28:53.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7869999999999999e-05 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.072883+0000", 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3877999999999998e-05 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4483.0:139 3.5 3:a50d5ea9:::rbd_data.11065aee08ca.000000000000000b:3 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.071729+0000", 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.43312248199999998, 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013297280000000001, 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4483", 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071729+0000", 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071729+0000", 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071730+0000", 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071732+0000", 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.491e-06 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071739+0000", 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0090000000000001e-06 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.071751+0000", 2024-09-11T09:28:53.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1528e-05 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.072908+0000", 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011571940000000001 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.072974+0000", 2024-09-11T09:28:53.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5748999999999997e-05 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.073059+0000", 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5164e-05 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:144 3.8 e27/26 3:156b4e14:::rbd_object_map.11064c3f4ab3:head v 27'6, mlcod=27'2)", 2024-09-11T09:28:53.570 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.074120+0000", 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.43073193500000001, 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00238097, 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074120+0000", 2024-09-11T09:28:53.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074120+0000", 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074120+0000", 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2899999999999999e-07 2024-09-11T09:28:53.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074125+0000", 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6539999999999998e-06 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074132+0000", 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.221e-06 2024-09-11T09:28:53.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074136+0000", 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3950000000000003e-06 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074172+0000", 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5224e-05 2024-09-11T09:28:53.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074185+0000", 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3402e-05 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076460+0000", 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022748740000000001 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076501+0000", 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0771e-05 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:145 3.8 e27/26 3:156b4e14:::rbd_object_map.11064c3f4ab3:head v 27'7, mlcod=27'3)", 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.074826+0000", 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.43002513199999998, 2024-09-11T09:28:53.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0044474989999999997, 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074826+0000", 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074826+0000", 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074827+0000", 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074839+0000", 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1824000000000001e-05 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074846+0000", 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2409999999999998e-06 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074850+0000", 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.827e-06 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074891+0000", 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0831000000000003e-05 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.074903+0000", 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2099000000000001e-05 2024-09-11T09:28:53.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079236+0000", 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0043328589999999997 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079274+0000", 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8241000000000002e-05 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4483.0:150 3.f 3:f3ecc0a5:::rbd_data.11064c3f4ab3.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15204352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.075285+0000", 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42956704000000001, 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040758699999999997, 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4483", 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 150 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075285+0000", 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075285+0000", 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075285+0000", 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8800000000000002e-07 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075365+0000", 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9895000000000007e-05 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075373+0000", 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9359999999999999e-06 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075377+0000", 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0450000000000002e-06 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075416+0000", 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9239999999999997e-05 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075500+0000", 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3901999999999997e-05 2024-09-11T09:28:53.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075638+0000", 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000137286 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076749+0000", 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011117760000000001 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079275+0000", 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025260119999999998 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079321+0000", 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5711000000000003e-05 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079360+0000", 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9279000000000001e-05 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:146 3.8 e27/26 3:156b4e14:::rbd_object_map.11064c3f4ab3:head v 27'8, mlcod=27'4)", 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.075451+0000", 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42940083400000001, 2024-09-11T09:28:53.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0038617510000000001, 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075451+0000", 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075451+0000", 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075451+0000", 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075462+0000", 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1001e-05 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075471+0000", 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1970000000000002e-06 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075476+0000", 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6210000000000003e-06 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075507+0000", 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1458000000000003e-05 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.075519+0000", 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1455e-05 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079277+0000", 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0037583849999999999 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079312+0000", 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5206000000000001e-05 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:147 3.8 e27/26 3:156b4e14:::rbd_object_map.11064c3f4ab3:head v 27'9, mlcod=27'4)", 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.076030+0000", 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42882146999999998, 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033087699999999999, 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076030+0000", 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076030+0000", 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076031+0000", 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4099999999999998e-07 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076035+0000", 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2629999999999997e-06 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076043+0000", 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7759999999999997e-06 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076047+0000", 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.782e-06 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076077+0000", 2024-09-11T09:28:53.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0504000000000001e-05 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076087+0000", 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0275000000000001e-05 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079314+0000", 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0032264080000000001 2024-09-11T09:28:53.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079339+0000", 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5120999999999999e-05 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:148 3.8 e27/26 3:156b4e14:::rbd_object_map.11064c3f4ab3:head v 27'10, mlcod=27'4)", 2024-09-11T09:28:53.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.076909+0000", 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42794232599999998, 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002664231, 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076909+0000", 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076909+0000", 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076910+0000", 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2500000000000005e-07 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076915+0000", 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076925+0000", 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0451999999999999e-05 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076932+0000", 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6610000000000003e-06 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076971+0000", 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9041000000000001e-05 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.076982+0000", 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1600000000000001e-05 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079547+0000", 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025647909999999999 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079573+0000", 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6205999999999999e-05 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:151 3.2 e27/21 3:44e5cd28:::rbd_data.11064c3f4ab3.0000000000000002:head v 27'2, mlcod=0'0)", 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.077924+0000", 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42692751200000001, 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001664334, 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.077924+0000", 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.077924+0000", 2024-09-11T09:28:53.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.077925+0000", 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6599999999999996e-07 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.078049+0000", 2024-09-11T09:28:53.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012413800000000001 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.078058+0000", 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8119999999999997e-06 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.078062+0000", 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5979999999999999e-06 2024-09-11T09:28:53.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.078100+0000", 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7759000000000002e-05 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.078113+0000", 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3318e-05 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079553+0000", 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001440223 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079588+0000", 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4919999999999998e-05 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4483.0:152 3.1d 3:bd32a276:::rbd_data.11064c3f4ab3.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15204352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.079597+0000", 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42525489100000002, 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002486523, 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4483", 2024-09-11T09:28:53.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 152 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079597+0000", 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079597+0000", 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079597+0000", 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5600000000000005e-07 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079617+0000", 2024-09-11T09:28:53.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.982e-05 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079626+0000", 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4649999999999994e-06 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079631+0000", 2024-09-11T09:28:53.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429999999999996e-06 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079669+0000", 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7700000000000002e-05 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079745+0000", 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6868999999999999e-05 2024-09-11T09:28:53.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.079878+0000", 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00013244299999999999 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.081974+0000", 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020966079999999998 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082014+0000", 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9391999999999998e-05 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082048+0000", 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4168999999999998e-05 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082083+0000", 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5157999999999998e-05 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:153 3.8 e27/26 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:head v 27'11, mlcod=27'9)", 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.080246+0000", 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.424605658, 2024-09-11T09:28:53.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0025372760000000002, 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080246+0000", 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080246+0000", 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080246+0000", 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3399999999999999e-07 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080291+0000", 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.473e-05 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080301+0000", 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6849999999999996e-06 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080307+0000", 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0750000000000003e-06 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080352+0000", 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5039e-05 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080365+0000", 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3383999999999999e-05 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082753+0000", 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0023877600000000001 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082783+0000", 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0069e-05 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4483.0:154 3.8 e27/26 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:head v 27'12, mlcod=27'9)", 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.080992+0000", 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.42385941500000002, 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018119810000000001, 2024-09-11T09:28:53.625 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080992+0000", 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080992+0000", 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.080993+0000", 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5000000000000003e-07 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.081051+0000", 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.876e-05 2024-09-11T09:28:53.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.081061+0000", 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0860000000000008e-06 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.081065+0000", 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1330000000000001e-06 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.081096+0000", 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.112e-05 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.081107+0000", 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1041e-05 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082789+0000", 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001682163 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.082804+0000", 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5128e-05 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4483.0:177 3.1d 3:bd32a276:::rbd_data.11064c3f4ab3.0000000000000003:head [stat,write 13535744~20480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:53.090749+0000", 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.41410243899999999, 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013150049999999999, 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4483", 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:53.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 177 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090749+0000", 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090749+0000", 2024-09-11T09:28:53.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090750+0000", 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3200000000000003e-07 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090771+0000", 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0622999999999999e-05 2024-09-11T09:28:53.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090781+0000", 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0389e-05 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090786+0000", 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1560000000000003e-06 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090836+0000", 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9916000000000001e-05 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.090922+0000", 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5790000000000004e-05 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:53.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.091019+0000", 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6838999999999999e-05 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.091972+0000", 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00095346099999999996 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.091991+0000", 2024-09-11T09:28:53.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9040999999999999e-05 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.092028+0000", 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7159000000000001e-05 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:53.092064+0000", 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5698999999999998e-05 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:53.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:53.638 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:53.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:28:53.640 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:53.640+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:53.740 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:53.740+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:53.760 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:53.760 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:53.760 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:53.760 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:53.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:28:53.782 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone1 from 0x65d3400 (overlap 0x65d3400) to 0x145d600 2024-09-11T09:28:53.841 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:53.841+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:53.883 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:53.884 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:53.884 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:53.884 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:53.884 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:53.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:28:53.977 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:53.977+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:54.043 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:54.044+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:54.046 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:54.046 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:28:54.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:35 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'21, mlcod=14'17)", 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.211857810000001, 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010916070000000001, 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833931+0000", 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1399999999999997e-07 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.833998+0000", 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6241000000000005e-05 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834010+0000", 2024-09-11T09:28:54.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2349e-05 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834025+0000", 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4962e-05 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834063+0000", 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7855000000000003e-05 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834076+0000", 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3352e-05 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.834987+0000", 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00091061400000000002 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.835022+0000", 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5620000000000001e-05 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4171.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'42, mlcod=14'39)", 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:30.859822+0000", 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.185966616999998, 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011026440000000001, 2024-09-11T09:28:54.052 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859822+0000", 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859822+0000", 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859823+0000", 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1659999999999999e-06 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859845+0000", 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1654e-05 2024-09-11T09:28:54.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859879+0000", 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3936000000000002e-05 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.859883+0000", 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7629999999999998e-06 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860094+0000", 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000211719 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860116+0000", 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1594e-05 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860895+0000", 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00077924899999999998 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:30.860925+0000", 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9563000000000001e-05 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.709953+0000", 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.3358357879999998, 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0081784709999999997, 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.058 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 11 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.709953+0000", 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.709953+0000", 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.709954+0000", 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.203e-06 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.709961+0000", 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5239999999999997e-06 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.709972+0000", 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1486e-05 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.709978+0000", 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.710024+0000", 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5398999999999997e-05 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.710028+0000", 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7009999999999998e-06 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.711809+0000", 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017813740000000001 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.711955+0000", 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000146193 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:28:54.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.712187+0000", 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00023182699999999999 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.712942+0000", 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000754544 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718007+0000", 2024-09-11T09:28:54.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.005065862 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718074+0000", 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7055000000000001e-05 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718131+0000", 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6889999999999999e-05 2024-09-11T09:28:54.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:12 3.1 3:846053a3:::rbd_header.11065aee08ca:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.718490+0000", 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.327299054, 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022284700000000002, 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 12 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718490+0000", 2024-09-11T09:28:54.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718490+0000", 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718491+0000", 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1379999999999999e-06 2024-09-11T09:28:54.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718497+0000", 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7220000000000002e-06 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718509+0000", 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1079e-05 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718517+0000", 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2889999999999998e-06 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718568+0000", 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1606e-05 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718678+0000", 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109639 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.718826+0000", 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014748800000000001 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.719479+0000", 2024-09-11T09:28:54.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065376400000000004 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.720641+0000", 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001161797 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.720676+0000", 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5358000000000003e-05 2024-09-11T09:28:54.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.720718+0000", 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.159e-05 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25 3.1 3:846053a3:::rbd_header.11065aee08ca:head [watch watch cookie 140373489986576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:54.073 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.733597+0000", 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.312191662, 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001283822, 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25 2024-09-11T09:28:54.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733597+0000", 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733597+0000", 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733598+0000", 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2600000000000002e-07 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733600+0000", 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.526e-06 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733607+0000", 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7580000000000003e-06 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733611+0000", 2024-09-11T09:28:54.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.692e-06 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733659+0000", 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8007999999999998e-05 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733744+0000", 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4903999999999999e-05 2024-09-11T09:28:54.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.733928+0000", 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018417800000000001 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.734476+0000", 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00054836499999999999 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.734763+0000", 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028686500000000003 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.734812+0000", 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8959000000000003e-05 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.734881+0000", 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8941000000000003e-05 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4385.0:12 3.1 3:846053a3:::rbd_header.11065aee08ca:head [watch watch cookie 140713194960688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.891233+0000", 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.1545558729999996, 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001179355, 2024-09-11T09:28:54.081 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4385", 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2255862569", 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 12 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891233+0000", 2024-09-11T09:28:54.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891233+0000", 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891234+0000", 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.37e-07 2024-09-11T09:28:54.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891237+0000", 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5159999999999999e-06 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891248+0000", 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0703999999999999e-05 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891260+0000", 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2717e-05 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891302+0000", 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2005000000000002e-05 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891347+0000", 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4966000000000003e-05 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.891460+0000", 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011254799999999999 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.892109+0000", 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00064915800000000005 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.892330+0000", 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00022117600000000001 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.892366+0000", 2024-09-11T09:28:54.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5902999999999998e-05 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.892412+0000", 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5924999999999998e-05 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:251 3.1 3:846053a3:::rbd_header.11065aee08ca:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:52.731619+0000", 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.3141694500000001, 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0014405080000000001, 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 251 2024-09-11T09:28:54.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731619+0000", 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731619+0000", 2024-09-11T09:28:54.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731620+0000", 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7100000000000001e-07 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731625+0000", 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.198e-06 2024-09-11T09:28:54.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731633+0000", 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9219999999999997e-06 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731639+0000", 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8139999999999997e-06 2024-09-11T09:28:54.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731697+0000", 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8539999999999999e-05 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.731788+0000", 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0833000000000003e-05 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.732065+0000", 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00027644699999999998 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.732710+0000", 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00064574599999999999 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.732952+0000", 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000241442 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.733003+0000", 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1413999999999999e-05 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.733060+0000", 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6480999999999997e-05 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4385.0:466 3.1 3:846053a3:::rbd_header.11065aee08ca:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:52.933000+0000", 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.1127891169999999, 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001186814, 2024-09-11T09:28:54.096 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4385", 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2255862569", 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 466 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933000+0000", 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933000+0000", 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933000+0000", 2024-09-11T09:28:54.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8299999999999996e-07 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933005+0000", 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4819999999999996e-06 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933016+0000", 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0910000000000001e-05 2024-09-11T09:28:54.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933023+0000", 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.396e-06 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933068+0000", 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4472999999999999e-05 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933120+0000", 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1955e-05 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933225+0000", 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010535300000000001 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.933639+0000", 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00041436700000000001 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934132+0000", 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00049284299999999997 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934161+0000", 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9200000000000002e-05 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934186+0000", 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5052e-05 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4385.0:468 3.1 3:846053a3:::rbd_header.11065aee08ca:head [watch unwatch cookie 140713194960688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:52.934365+0000", 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.1114233689999999, 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0026081170000000001, 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.104 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4385", 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2255862569", 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 468 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934365+0000", 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934365+0000", 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934366+0000", 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.34e-07 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934368+0000", 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934372+0000", 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5700000000000003e-06 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934375+0000", 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.095e-06 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934561+0000", 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000186019 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934601+0000", 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9963999999999999e-05 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934695+0000", 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3592999999999997e-05 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.935060+0000", 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00036550999999999999 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.936875+0000", 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018143580000000001 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.936902+0000", 2024-09-11T09:28:54.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7345e-05 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.936973+0000", 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1228000000000003e-05 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.111 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4385.0:469 3.1 3:846053a3:::rbd_header.11065aee08ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:52.934602+0000", 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.1111862509999999, 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0024960579999999998, 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4385", 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2255862569", 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 469 2024-09-11T09:28:54.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934602+0000", 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934602+0000", 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934603+0000", 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1099999999999995e-07 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934606+0000", 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.022e-06 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934611+0000", 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4369999999999998e-06 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934615+0000", 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6679999999999999e-06 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934882+0000", 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00026645400000000001 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934926+0000", 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4888000000000001e-05 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.937010+0000", 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.002083054 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.937051+0000", 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1857999999999998e-05 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.937098+0000", 2024-09-11T09:28:54.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7066e-05 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:305 3.1 3:846053a3:::rbd_header.11065aee08ca:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:52.934766+0000", 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.111022357, 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0024520290000000001, 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.118 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 305 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934766+0000", 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934766+0000", 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934767+0000", 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9699999999999995e-07 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934773+0000", 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3199999999999996e-06 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934790+0000", 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.624e-05 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934797+0000", 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1720000000000001e-06 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934952+0000", 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00015532400000000001 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:28:54.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.934993+0000", 2024-09-11T09:28:54.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0917999999999997e-05 2024-09-11T09:28:54.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.937124+0000", 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021305370000000001 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.937160+0000", 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6275000000000001e-05 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.937218+0000", 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8446000000000003e-05 2024-09-11T09:28:54.125 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:306 3.1 3:846053a3:::rbd_header.11065aee08ca:head [watch unwatch cookie 140373489986576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:52.945606+0000", 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.1001827710000001, 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013037979999999999, 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.126 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 306 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945606+0000", 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945606+0000", 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945608+0000", 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.897e-06 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945617+0000", 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5429999999999999e-06 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945646+0000", 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8730000000000001e-05 2024-09-11T09:28:54.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945663+0000", 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6877000000000001e-05 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945827+0000", 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00016357500000000001 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.945950+0000", 2024-09-11T09:28:54.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012342200000000001 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.946031+0000", 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1151999999999998e-05 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.946412+0000", 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00038088599999999998 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.946836+0000", 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00042450799999999998 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.946874+0000", 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7628e-05 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:52.946910+0000", 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5580000000000002e-05 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.131 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4483.0:125 3.1f 3:f82ee220:::rbd_data.11065aee08ca.0000000000000004:3 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.071706+0000", 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.97408283600000001, 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001102132, 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4483", 2024-09-11T09:28:54.132 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 125 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071706+0000", 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071706+0000", 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071706+0000", 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2999999999999999e-07 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071707+0000", 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.234e-06 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071710+0000", 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0259999999999999e-06 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071713+0000", 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5739999999999998e-06 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072620+0000", 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090678000000000004 2024-09-11T09:28:54.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072691+0000", 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1605000000000003e-05 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072808+0000", 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011668299999999999 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4483.0:132 3.6 3:663f53ba:::rbd_data.11065aee08ca.0000000000000018:3 [sparse-read 1048576~28672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.071716+0000", 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.97407285499999996, 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011983709999999999, 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4483", 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:54.137 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 132 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071716+0000", 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071716+0000", 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071716+0000", 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0100000000000001e-07 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071718+0000", 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.978e-06 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071721+0000", 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0079999999999998e-06 2024-09-11T09:28:54.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071724+0000", 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.661e-06 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072629+0000", 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000905242 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072702+0000", 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3509000000000004e-05 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072914+0000", 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00021177199999999999 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4483.0:134 3.1f 3:f82ee220:::rbd_data.11065aee08ca.0000000000000004:3 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.141 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.071727+0000", 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.97406199199999999, 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001376588, 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4483", 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 134 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071727+0000", 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071727+0000", 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071727+0000", 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071728+0000", 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6800000000000009e-07 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071731+0000", 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6229999999999999e-06 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071734+0000", 2024-09-11T09:28:54.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5600000000000001e-06 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072861+0000", 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011269089999999999 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072962+0000", 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010137 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.073103+0000", 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014095000000000001 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4483.0:141 3.6 3:663f53ba:::rbd_data.11065aee08ca.0000000000000018:3 [sparse-read 1572864~40960] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.071737+0000", 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.97405139500000004, 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0015137460000000001, 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.146 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4483", 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 141 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071737+0000", 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071737+0000", 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071737+0000", 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.36e-07 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071745+0000", 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9840000000000001e-06 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071749+0000", 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5200000000000002e-06 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.071753+0000", 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7780000000000001e-06 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.072958+0000", 2024-09-11T09:28:54.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0012056910000000001 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.073005+0000", 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6109000000000002e-05 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.073251+0000", 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00024642799999999998 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.150 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4483.0:155 3.1 3:877a7a86:::rbd_data.11064c3f4ab3.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15204352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.080022+0000", 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.96576621299999998, 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.003003744, 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4483", 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:54.151 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 155 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080022+0000", 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080022+0000", 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080023+0000", 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0599999999999999e-07 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080162+0000", 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013825799999999999 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080173+0000", 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1528999999999999e-05 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080179+0000", 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4319999999999998e-06 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080236+0000", 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7479000000000003e-05 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080375+0000", 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013941299999999999 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.080621+0000", 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000245293 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.081718+0000", 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001097521 2024-09-11T09:28:54.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.082949+0000", 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0012311189999999999 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.082994+0000", 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5024999999999997e-05 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.083026+0000", 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1769e-05 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:393 3.1 3:846053a3:::rbd_header.11065aee08ca:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.746649+0000", 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.29913963100000002, 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021174750000000002, 2024-09-11T09:28:54.157 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 393 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746649+0000", 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746649+0000", 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746651+0000", 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1799999999999999e-06 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746661+0000", 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6190000000000006e-06 2024-09-11T09:28:54.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746680+0000", 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9315000000000001e-05 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746690+0000", 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5470000000000002e-06 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746768+0000", 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7999999999999999e-05 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.746772+0000", 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6890000000000003e-06 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.748566+0000", 2024-09-11T09:28:54.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017934649999999999 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.748636+0000", 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0370000000000006e-05 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.748766+0000", 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013029 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.162 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4483.0:281 3.1 3:877a7a86:::rbd_data.11064c3f4ab3.0000000000000006:head [stat,truncate 2621440] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:53.799181+0000", 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.24660721899999999, 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001335227, 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4483", 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1829778685", 2024-09-11T09:28:54.163 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 281 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799181+0000", 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799181+0000", 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799183+0000", 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7409999999999999e-06 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799191+0000", 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799208+0000", 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7138000000000001e-05 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799219+0000", 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0710000000000001e-05 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799275+0000", 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6836999999999998e-05 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799332+0000", 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6592000000000003e-05 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.799448+0000", 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000115863 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.800213+0000", 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00076555199999999996 2024-09-11T09:28:54.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.800409+0000", 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00019512800000000001 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.800469+0000", 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0180999999999999e-05 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:53.800517+0000", 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7963000000000001e-05 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:507 3.1 3:846053a3:::rbd_header.11065aee08ca:head [watch watch cookie 140373489965200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.000451+0000", 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.045337757999999999, 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011123820000000001, 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:54.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 507 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000451+0000", 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000451+0000", 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000452+0000", 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1069999999999999e-06 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000457+0000", 2024-09-11T09:28:54.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5079999999999999e-06 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000468+0000", 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1674999999999999e-05 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000476+0000", 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000519+0000", 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3307999999999999e-05 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000564+0000", 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5090999999999999e-05 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.000661+0000", 2024-09-11T09:28:54.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6848999999999994e-05 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.001385+0000", 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00072402499999999999 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.001486+0000", 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000100989 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.001519+0000", 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3222000000000001e-05 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.001563+0000", 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3921999999999997e-05 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:54.175 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:54.176 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-11T09:28:54.177 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:28:54.178 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:54.177+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:54.278 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:54.278+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:54.332 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:54.333 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:54.333 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:54.333 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:54.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:28:54.379 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:54.380+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:54.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:28:54.491 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:54.491 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:54.491 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:54.491 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:54.491 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:54.491 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:54.491+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:54.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:28:54.560 INFO:teuthology.orchestra.run.smithi107.stdout:117 read 0x1871000 thru 0x187c5a9 (0xb5aa bytes) 2024-09-11T09:28:54.562 INFO:teuthology.orchestra.run.smithi107.stdout:119 read 0x6229400 thru 0x62377f3 (0xe3f4 bytes) 2024-09-11T09:28:54.562 INFO:teuthology.orchestra.run.smithi107.stdout:120 punch from 0x5cb7000 to 0x5cc1400, (0xa400 bytes) 2024-09-11T09:28:54.584 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:54.585 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:54.585 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:54.585+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:54.590 INFO:teuthology.orchestra.run.smithi107.stdout:121 trunc from 0x65d3400 to 0x8147400 2024-09-11T09:28:54.599 INFO:teuthology.orchestra.run.smithi107.stdout:123 read 0x4a1b000 thru 0x4a1e8b9 (0x38ba bytes) 2024-09-11T09:28:54.603 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:54.603 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:54.606 INFO:teuthology.orchestra.run.smithi107.stdout:125 trunc from 0x8147400 to 0xcb66a00 2024-09-11T09:28:54.615 INFO:teuthology.orchestra.run.smithi107.stdout:127 write 0x52eb200 thru 0x52f5e36 (0xac37 bytes) 2024-09-11T09:28:54.617 INFO:teuthology.orchestra.run.smithi107.stdout:129 read 0xa357e00 thru 0xa359e79 (0x207a bytes) 2024-09-11T09:28:54.617 INFO:teuthology.orchestra.run.smithi107.stdout:130 punch from 0x72cfe00 to 0x72d2400, (0x2600 bytes) 2024-09-11T09:28:54.620 INFO:teuthology.orchestra.run.smithi107.stdout:131 read 0x2ad8000 thru 0x2ae32ba (0xb2bb bytes) 2024-09-11T09:28:54.624 INFO:teuthology.orchestra.run.smithi107.stdout:132 trunc from 0xcb66a00 to 0x9995600 2024-09-11T09:28:54.625 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:54.625 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:54.636 INFO:teuthology.orchestra.run.smithi107.stdout:134 read 0x18ff600 thru 0x190cfa7 (0xd9a8 bytes) 2024-09-11T09:28:54.643 INFO:teuthology.orchestra.run.smithi107.stdout:135 read 0x1b1e600 thru 0x1b2b396 (0xcd97 bytes) 2024-09-11T09:28:54.643 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:54.644 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:54.661 INFO:teuthology.orchestra.run.smithi107.stdout:136 write 0xed49a00 thru 0xed4ddee (0x43ef bytes) 2024-09-11T09:28:54.663 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:54.663 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:54.664 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:54.664 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:54.666 INFO:teuthology.orchestra.run.smithi107.stdout:137 read 0x2e46200 thru 0x2e51f86 (0xbd87 bytes) 2024-09-11T09:28:54.667 INFO:teuthology.orchestra.run.smithi107.stdout:138 read 0x50e5400 thru 0x50f30ea (0xdceb bytes) 2024-09-11T09:28:54.667 INFO:teuthology.orchestra.run.smithi107.stdout:139 punch from 0xd458a00 to 0xd460400, (0x7a00 bytes) 2024-09-11T09:28:54.667 INFO:teuthology.orchestra.run.smithi107.stdout:140 punch from 0x4812000 to 0x4819800, (0x7800 bytes) 2024-09-11T09:28:54.668 INFO:teuthology.orchestra.run.smithi107.stdout:143 read 0x5bb6200 thru 0x5bbf44c (0x924d bytes) 2024-09-11T09:28:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:145 read 0x58ce200 thru 0x58d7bd0 (0x99d1 bytes) 2024-09-11T09:28:54.674 INFO:teuthology.orchestra.run.smithi107.stdout:147 punch from 0x8085c00 to 0x8094c00, (0xf000 bytes) 2024-09-11T09:28:54.674 INFO:teuthology.orchestra.run.smithi107.stdout:148 read 0x8ca2e00 thru 0x8cad102 (0xa303 bytes) 2024-09-11T09:28:54.674 INFO:teuthology.orchestra.run.smithi107.stdout:150 write 0x697a600 thru 0x698209f (0x7aa0 bytes) 2024-09-11T09:28:54.674 INFO:teuthology.orchestra.run.smithi107.stdout:151 read 0x4c5ae00 thru 0x4c5ea82 (0x3c83 bytes) 2024-09-11T09:28:54.676 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:28:54.685 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:54.685 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:54.685 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:54.685+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:54.695 INFO:teuthology.orchestra.run.smithi107.stdout:152 trunc from 0xed4de00 to 0x25bbe00 2024-09-11T09:28:54.732 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:28:54.770 INFO:teuthology.orchestra.run.smithi107.stdout:153 read 0x114a800 thru 0x1159032 (0xe833 bytes) 2024-09-11T09:28:54.777 INFO:teuthology.orchestra.run.smithi107.stdout:154 trunc from 0x25bbe00 to 0x62ea000 2024-09-11T09:28:54.782 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:54.783+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:54.807 INFO:teuthology.orchestra.run.smithi107.stdout:158 write 0x845d800 thru 0x84669ca (0x91cb bytes) 2024-09-11T09:28:54.808 INFO:teuthology.orchestra.run.smithi107.stdout:160 write 0x1c4f600 thru 0x1c5b7f6 (0xc1f7 bytes) 2024-09-11T09:28:54.808 INFO:teuthology.orchestra.run.smithi107.stdout:165 read 0x7c39a00 thru 0x7c3bc3b (0x223c bytes) 2024-09-11T09:28:54.814 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:54.814 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:54.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:28:54.849 INFO:teuthology.orchestra.run.smithi107.stdout:167 write 0xa0a4400 thru 0xa0a49b6 (0x5b7 bytes) 2024-09-11T09:28:54.849 INFO:teuthology.orchestra.run.smithi107.stdout:172 write 0x31cd000 thru 0x31d4bfe (0x7bff bytes) 2024-09-11T09:28:54.873 INFO:teuthology.orchestra.run.smithi107.stdout:173 write 0xdc25400 thru 0xdc2fb2c (0xa72d bytes) 2024-09-11T09:28:54.873 INFO:teuthology.orchestra.run.smithi107.stdout:175 read 0x7a2d600 thru 0x7a3256f (0x4f70 bytes) 2024-09-11T09:28:54.873 INFO:teuthology.orchestra.run.smithi107.stdout:176 read 0x75f0a00 thru 0x75f2474 (0x1a75 bytes) 2024-09-11T09:28:54.874 INFO:teuthology.orchestra.run.smithi107.stdout:177 trunc from 0xdc2fc00 to 0xd34de00 2024-09-11T09:28:54.896 INFO:teuthology.orchestra.run.smithi107.stdout:178 write 0x4aa4c00 thru 0x4aadcc7 (0x90c8 bytes) 2024-09-11T09:28:54.897 INFO:teuthology.orchestra.run.smithi107.stdout:179 read 0x4624600 thru 0x462d7f5 (0x91f6 bytes) 2024-09-11T09:28:54.897 INFO:teuthology.orchestra.run.smithi107.stdout:180 read 0xbe40400 thru 0xbe40bb7 (0x7b8 bytes) 2024-09-11T09:28:54.897 INFO:teuthology.orchestra.run.smithi107.stdout:183 punch from 0x6485c00 to 0x6493200, (0xd600 bytes) 2024-09-11T09:28:54.897 INFO:teuthology.orchestra.run.smithi107.stdout:186 punch from 0xb13ce00 to 0xb144c00, (0x7e00 bytes) 2024-09-11T09:28:54.897 INFO:teuthology.orchestra.run.smithi107.stdout:188 write 0xe17600 thru 0xe20c83 (0x9684 bytes) 2024-09-11T09:28:54.897 INFO:teuthology.orchestra.run.smithi107.stdout:189 read 0x942f000 thru 0x943d612 (0xe613 bytes) 2024-09-11T09:28:54.908 INFO:teuthology.orchestra.run.smithi107.stdout:190 trunc from 0xd34de00 to 0x2e3b000 2024-09-11T09:28:54.911 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:54.912+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:54.959 INFO:teuthology.orchestra.run.smithi107.stdout:192 punch from 0x2b57600 to 0x2b5c000, (0x4a00 bytes) 2024-09-11T09:28:54.964 INFO:teuthology.orchestra.run.smithi107.stdout:193 punch from 0x2a7f400 to 0x2a80600, (0x1200 bytes) 2024-09-11T09:28:54.984 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:54.985+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:54.984 INFO:teuthology.orchestra.run.smithi107.stdout:198 write 0xac81a00 thru 0xac853f9 (0x39fa bytes) 2024-09-11T09:28:54.985 INFO:teuthology.orchestra.run.smithi107.stdout:199 read 0x126f800 thru 0x1279113 (0x9914 bytes) 2024-09-11T09:28:54.986 INFO:teuthology.orchestra.run.smithi107.stdout:202 punch from 0x79b8600 to 0x79b9400, (0xe00 bytes) 2024-09-11T09:28:54.986 INFO:teuthology.orchestra.run.smithi107.stdout:204 write 0x4336000 thru 0x433a99f (0x49a0 bytes) 2024-09-11T09:28:54.986 INFO:teuthology.orchestra.run.smithi107.stdout:209 read 0x3ac4a00 thru 0x3ad03c0 (0xb9c1 bytes) 2024-09-11T09:28:54.986 INFO:teuthology.orchestra.run.smithi107.stdout:211 read 0x4eed200 thru 0x4eeff0a (0x2d0b bytes) 2024-09-11T09:28:54.986 INFO:teuthology.orchestra.run.smithi107.stdout:212 read 0x2c46a00 thru 0x2c4c255 (0x5856 bytes) 2024-09-11T09:28:54.990 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:54.990 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:54.993 INFO:teuthology.orchestra.run.smithi107.stdout:213 trunc from 0xac85400 to 0x4d27200 2024-09-11T09:28:55.000 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:28:55.015 INFO:teuthology.orchestra.run.smithi107.stdout:214 read 0x479800 thru 0x485f3a (0xc73b bytes) 2024-09-11T09:28:55.023 INFO:teuthology.orchestra.run.smithi107.stdout:215 read 0x17e3200 thru 0x17ec003 (0x8e04 bytes) 2024-09-11T09:28:55.023 INFO:teuthology.orchestra.run.smithi107.stdout:217 read 0xabd800 thru 0xacb9f1 (0xe1f2 bytes) 2024-09-11T09:28:55.030 INFO:teuthology.orchestra.run.smithi107.stdout:219 trunc from 0x4d27200 to 0xa0c9400 2024-09-11T09:28:55.050 INFO:teuthology.orchestra.run.smithi107.stdout:221 trunc from 0xa0c9400 to 0x4aece00 2024-09-11T09:28:55.061 INFO:teuthology.orchestra.run.smithi107.stdout:222 read 0x14d3e00 thru 0x14d54a4 (0x16a5 bytes) 2024-09-11T09:28:55.063 INFO:teuthology.orchestra.run.smithi107.stdout:228 read 0x3b02a00 thru 0x3b0eb67 (0xc168 bytes) 2024-09-11T09:28:55.063 INFO:teuthology.orchestra.run.smithi107.stdout:232 read 0x38de800 thru 0x38e3890 (0x5091 bytes) 2024-09-11T09:28:55.064 INFO:teuthology.orchestra.run.smithi107.stdout:236 write 0x2c47200 thru 0x2c4f001 (0x7e02 bytes) 2024-09-11T09:28:55.082 INFO:teuthology.orchestra.run.smithi107.stdout:237 write 0x7d9ea00 thru 0x7da5f9b (0x759c bytes) 2024-09-11T09:28:55.084 INFO:teuthology.orchestra.run.smithi107.stdout:239 write 0x58b9c00 thru 0x58bda98 (0x3e99 bytes) 2024-09-11T09:28:55.084 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:55.085+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:55.110 INFO:teuthology.orchestra.run.smithi107.stdout:241 write 0xde17200 thru 0xde19002 (0x1e03 bytes) 2024-09-11T09:28:55.111 INFO:teuthology.orchestra.run.smithi107.stdout:242 punch from 0x6d18200 to 0x6d24600, (0xc400 bytes) 2024-09-11T09:28:55.116 INFO:teuthology.orchestra.run.smithi107.stdout:246 punch from 0x1d2d800 to 0x1d2f400, (0x1c00 bytes) 2024-09-11T09:28:55.116 INFO:teuthology.orchestra.run.smithi107.stdout:247 write 0xc225600 thru 0xc228576 (0x2f77 bytes) 2024-09-11T09:28:55.117 INFO:teuthology.orchestra.run.smithi107.stdout:248 trunc from 0xde19200 to 0xd557c00 2024-09-11T09:28:55.144 INFO:teuthology.orchestra.run.smithi107.stdout:249 trunc from 0xd557c00 to 0x54eae00 2024-09-11T09:28:55.160 INFO:teuthology.orchestra.run.smithi107.stdout:251 write 0x2c10000 thru 0x2c1a9e5 (0xa9e6 bytes) 2024-09-11T09:28:55.160 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.160 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:28:55.175 INFO:teuthology.orchestra.run.smithi107.stdout:255 write 0x7093200 thru 0x7099f16 (0x6d17 bytes) 2024-09-11T09:28:55.198 INFO:teuthology.orchestra.run.smithi107.stdout:256 write 0xed85600 thru 0xed946f9 (0xf0fa bytes) 2024-09-11T09:28:55.211 INFO:teuthology.orchestra.run.smithi107.stdout:257 trunc from 0xed94800 to 0x4847e00 2024-09-11T09:28:55.211 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:55.212+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:55.246 INFO:teuthology.orchestra.run.smithi107.stdout:260 write 0x8d44800 thru 0x8d49e9a (0x569b bytes) 2024-09-11T09:28:55.265 INFO:teuthology.orchestra.run.smithi107.stdout:261 write 0xe395200 thru 0xe39b44e (0x624f bytes) 2024-09-11T09:28:55.265 INFO:teuthology.orchestra.run.smithi107.stdout:262 punch from 0xb688000 to 0xb688600, (0x600 bytes) 2024-09-11T09:28:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:264 write 0xc1e6200 thru 0xc1f537e (0xf17f bytes) 2024-09-11T09:28:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:268 write 0x57e3800 thru 0x57e7206 (0x3a07 bytes) 2024-09-11T09:28:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:269 read 0xcf51200 thru 0xcf5c613 (0xb414 bytes) 2024-09-11T09:28:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:270 read 0xbd6e800 thru 0xbd74229 (0x5a2a bytes) 2024-09-11T09:28:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:273 write 0xa5b0200 thru 0xa5c00ff (0xff00 bytes) 2024-09-11T09:28:55.267 INFO:teuthology.orchestra.run.smithi107.stdout:275 read 0xfc3800 thru 0xfcd803 (0xa004 bytes) 2024-09-11T09:28:55.269 INFO:teuthology.orchestra.run.smithi107.stdout:278 trunc from 0xe39b600 to 0xc8ef600 2024-09-11T09:28:55.286 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:55.286+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:55.289 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.289 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.291 INFO:teuthology.orchestra.run.smithi107.stdout:283 write 0x6ef5200 thru 0x6efa5c7 (0x53c8 bytes) 2024-09-11T09:28:55.292 INFO:teuthology.orchestra.run.smithi107.stdout:288 read 0xac5d000 thru 0xac6425e (0x725f bytes) 2024-09-11T09:28:55.292 INFO:teuthology.orchestra.run.smithi107.stdout:291 read 0xc88dc00 thru 0xc898f34 (0xb335 bytes) 2024-09-11T09:28:55.292 INFO:teuthology.orchestra.run.smithi107.stdout:292 read 0xe7e400 thru 0xe8ddc5 (0xf9c6 bytes) 2024-09-11T09:28:55.296 INFO:teuthology.orchestra.run.smithi107.stdout:293 read 0xc690400 thru 0xc697b8a (0x778b bytes) 2024-09-11T09:28:55.299 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:28:55.309 INFO:teuthology.orchestra.run.smithi107.stdout:296 write 0xcdc1e00 thru 0xcdcddc8 (0xbfc9 bytes) 2024-09-11T09:28:55.309 INFO:teuthology.orchestra.run.smithi107.stdout:298 write 0x8b02a00 thru 0x8b0f7b1 (0xcdb2 bytes) 2024-09-11T09:28:55.314 INFO:teuthology.orchestra.run.smithi107.stdout:299 read 0x9452a00 thru 0x945d87e (0xae7f bytes) 2024-09-11T09:28:55.314 INFO:teuthology.orchestra.run.smithi107.stdout:300 punch from 0xbd9a00 to 0xbe9800, (0xfe00 bytes) 2024-09-11T09:28:55.315 INFO:teuthology.orchestra.run.smithi107.stdout:301 write 0x536da00 thru 0x537b519 (0xdb1a bytes) 2024-09-11T09:28:55.315 INFO:teuthology.orchestra.run.smithi107.stdout:306 punch from 0x7c1a400 to 0x7c21e00, (0x7a00 bytes) 2024-09-11T09:28:55.315 INFO:teuthology.orchestra.run.smithi107.stdout:310 write 0xb2a9600 thru 0xb2b0da3 (0x77a4 bytes) 2024-09-11T09:28:55.315 INFO:teuthology.orchestra.run.smithi107.stdout:311 punch from 0xa31b000 to 0xa320200, (0x5200 bytes) 2024-09-11T09:28:55.337 INFO:teuthology.orchestra.run.smithi107.stdout:316 write 0xe9c7800 thru 0xe9d64ca (0xeccb bytes) 2024-09-11T09:28:55.357 INFO:teuthology.orchestra.run.smithi107.stdout:319 trunc from 0xe9d6600 to 0x21eda00 2024-09-11T09:28:55.387 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:55.387+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:55.398 INFO:teuthology.orchestra.run.smithi107.stdout:320 trunc from 0x21eda00 to 0x253dc00 2024-09-11T09:28:55.416 INFO:teuthology.orchestra.run.smithi107.stdout:322 punch from 0x11400 to 0x1ca00, (0xb600 bytes) 2024-09-11T09:28:55.421 INFO:teuthology.orchestra.run.smithi107.stdout:325 write 0x13b1600 thru 0x13c0a0d (0xf40e bytes) 2024-09-11T09:28:55.422 INFO:teuthology.orchestra.run.smithi107.stdout:328 punch from 0x1e7b800 to 0x1e84400, (0x8c00 bytes) 2024-09-11T09:28:55.423 INFO:teuthology.orchestra.run.smithi107.stdout:329 read 0x203be00 thru 0x2049349 (0xd54a bytes) 2024-09-11T09:28:55.428 INFO:teuthology.orchestra.run.smithi107.stdout:330 write 0x47200 thru 0x55d4a (0xeb4b bytes) 2024-09-11T09:28:55.439 INFO:teuthology.orchestra.run.smithi107.stdout:332 clone 3 order 20 su 32768 sc 5 2024-09-11T09:28:55.457 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.457 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:28:55.511 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:55.511+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:55.569 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.570 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.578 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:28:55.620 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:55.620+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:55.690 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:55.690+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:55.722 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:28:55.723 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.723 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.734 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:28:55.791 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:55.791+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:55.796 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone2 from 0x253dc00 (overlap 0x21eda00) to 0x10f6c00 2024-09-11T09:28:55.847 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.847 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:28:55.891 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:55.892+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:55.964 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:55.965 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:55.972 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:28:56.015 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:56.016+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:56.077 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:56.078 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:56.086 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:28:56.118 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-11T09:28:56.126 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:56.127+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:56.193 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:56.194+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:56.230 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:56.230 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:56.235 INFO:teuthology.orchestra.run.smithi107.stdout:333 read 0x191be00 thru 0x1926c81 (0xae82 bytes) 2024-09-11T09:28:56.239 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:28:56.260 INFO:teuthology.orchestra.run.smithi107.stdout:334 write 0xb5eca00 thru 0xb5f31f1 (0x67f2 bytes) 2024-09-11T09:28:56.281 INFO:teuthology.orchestra.run.smithi107.stdout:338 write 0xdf51c00 thru 0xdf5a2ed (0x86ee bytes) 2024-09-11T09:28:56.282 INFO:teuthology.orchestra.run.smithi107.stdout:339 read 0x108be00 thru 0x1097629 (0xb82a bytes) 2024-09-11T09:28:56.283 INFO:teuthology.orchestra.run.smithi107.stdout:340 read 0x8861200 thru 0x8870d13 (0xfb14 bytes) 2024-09-11T09:28:56.294 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:56.294+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:56.297 INFO:teuthology.orchestra.run.smithi107.stdout:341 trunc from 0xdf5a400 to 0x224fc00 2024-09-11T09:28:56.342 INFO:teuthology.orchestra.run.smithi107.stdout:342 read 0x162a200 thru 0x162c4f3 (0x22f4 bytes) 2024-09-11T09:28:56.350 INFO:teuthology.orchestra.run.smithi107.stdout:343 trunc from 0x224fc00 to 0x74d5600 2024-09-11T09:28:56.353 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:56.353 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:56.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:28:56.406 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:56.407+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:56.412 INFO:teuthology.orchestra.run.smithi107.stdout:345 punch from 0x398e00 to 0x3a8c00, (0xfe00 bytes) 2024-09-11T09:28:56.413 INFO:teuthology.orchestra.run.smithi107.stdout:346 read 0x1d1a200 thru 0x1d1e55e (0x435f bytes) 2024-09-11T09:28:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:348 write 0xa516a00 thru 0xa5222cc (0xb8cd bytes) 2024-09-11T09:28:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:349 read 0x713e000 thru 0x7146ac6 (0x8ac7 bytes) 2024-09-11T09:28:56.442 INFO:teuthology.orchestra.run.smithi107.stdout:350 trunc from 0xa522400 to 0xa752200 2024-09-11T09:28:56.474 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:56.474 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:56.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:28:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:351 write 0xd889600 thru 0xd891f16 (0x8917 bytes) 2024-09-11T09:28:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:352 read 0x2a7f800 thru 0x2a875c6 (0x7dc7 bytes) 2024-09-11T09:28:56.487 INFO:teuthology.orchestra.run.smithi107.stdout:354 punch from 0xcc91200 to 0xcc9e400, (0xd200 bytes) 2024-09-11T09:28:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:357 punch from 0x4332400 to 0x4332c00, (0x800 bytes) 2024-09-11T09:28:56.488 INFO:teuthology.orchestra.run.smithi107.stdout:358 write 0x5856e00 thru 0x58597c8 (0x29c9 bytes) 2024-09-11T09:28:56.502 INFO:teuthology.orchestra.run.smithi107.stdout:359 trunc from 0xd892000 to 0x1735600 2024-09-11T09:28:56.524 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:56.525+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:56.546 INFO:teuthology.orchestra.run.smithi107.stdout:360 read 0x125fe00 thru 0x12617a3 (0x19a4 bytes) 2024-09-11T09:28:56.575 INFO:teuthology.orchestra.run.smithi107.stdout:362 write 0xe756000 thru 0xe759f76 (0x3f77 bytes) 2024-09-11T09:28:56.576 INFO:teuthology.orchestra.run.smithi107.stdout:365 read 0x334ac00 thru 0x3354014 (0x9415 bytes) 2024-09-11T09:28:56.577 INFO:teuthology.orchestra.run.smithi107.stdout:366 write 0x901b400 thru 0x902162f (0x6230 bytes) 2024-09-11T09:28:56.577 INFO:teuthology.orchestra.run.smithi107.stdout:370 write 0x7b96c00 thru 0x7b9839f (0x17a0 bytes) 2024-09-11T09:28:56.577 INFO:teuthology.orchestra.run.smithi107.stdout:371 read 0x61ee400 thru 0x61f5fa5 (0x7ba6 bytes) 2024-09-11T09:28:56.577 INFO:teuthology.orchestra.run.smithi107.stdout:372 punch from 0x6658200 to 0x665c600, (0x4400 bytes) 2024-09-11T09:28:56.577 INFO:teuthology.orchestra.run.smithi107.stdout:373 read 0x3bbe600 thru 0x3bcbdab (0xd7ac bytes) 2024-09-11T09:28:56.578 INFO:teuthology.orchestra.run.smithi107.stdout:375 read 0x7a58000 thru 0x7a5c70a (0x470b bytes) 2024-09-11T09:28:56.578 INFO:teuthology.orchestra.run.smithi107.stdout:376 read 0x5efa200 thru 0x5efef0f (0x4d10 bytes) 2024-09-11T09:28:56.584 INFO:teuthology.orchestra.run.smithi107.stdout:378 trunc from 0xe75a000 to 0xa0e7800 2024-09-11T09:28:56.597 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:56.598+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:56.621 INFO:teuthology.orchestra.run.smithi107.stdout:379 write 0xc931800 thru 0xc936424 (0x4c25 bytes) 2024-09-11T09:28:56.621 INFO:teuthology.orchestra.run.smithi107.stdout:380 read 0x3bec800 thru 0x3bf3f53 (0x7754 bytes) 2024-09-11T09:28:56.621 INFO:teuthology.orchestra.run.smithi107.stdout:382 read 0x64c1200 thru 0x64c24d6 (0x12d7 bytes) 2024-09-11T09:28:56.622 INFO:teuthology.orchestra.run.smithi107.stdout:383 write 0x47c6c00 thru 0x47d3986 (0xcd87 bytes) 2024-09-11T09:28:56.622 INFO:teuthology.orchestra.run.smithi107.stdout:386 punch from 0x2d9e400 to 0x2dabc00, (0xd800 bytes) 2024-09-11T09:28:56.622 INFO:teuthology.orchestra.run.smithi107.stdout:387 write 0x7271a00 thru 0x7272824 (0xe25 bytes) 2024-09-11T09:28:56.622 INFO:teuthology.orchestra.run.smithi107.stdout:388 read 0x7a2bc00 thru 0x7a2d3c5 (0x17c6 bytes) 2024-09-11T09:28:56.622 INFO:teuthology.orchestra.run.smithi107.stdout:389 read 0x560d400 thru 0x5615ee1 (0x8ae2 bytes) 2024-09-11T09:28:56.636 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:56.636 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:56.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:28:56.647 INFO:teuthology.orchestra.run.smithi107.stdout:392 write 0xcfc5600 thru 0xcfd279e (0xd19f bytes) 2024-09-11T09:28:56.647 INFO:teuthology.orchestra.run.smithi107.stdout:394 read 0xcb9fc00 thru 0xcba3f47 (0x4348 bytes) 2024-09-11T09:28:56.647 INFO:teuthology.orchestra.run.smithi107.stdout:398 write 0xc55400 thru 0xc5a5a1 (0x51a2 bytes) 2024-09-11T09:28:56.648 INFO:teuthology.orchestra.run.smithi107.stdout:400 read 0x326e00 thru 0x33174b (0xa94c bytes) 2024-09-11T09:28:56.648 INFO:teuthology.orchestra.run.smithi107.stdout:401 write 0x1656200 thru 0x1664f46 (0xed47 bytes) 2024-09-11T09:28:56.649 INFO:teuthology.orchestra.run.smithi107.stdout:403 punch from 0x604ae00 to 0x6050c00, (0x5e00 bytes) 2024-09-11T09:28:56.650 INFO:teuthology.orchestra.run.smithi107.stdout:404 punch from 0x8cf0c00 to 0x8cfd400, (0xc800 bytes) 2024-09-11T09:28:56.650 INFO:teuthology.orchestra.run.smithi107.stdout:405 punch from 0x282b200 to 0x2838c00, (0xda00 bytes) 2024-09-11T09:28:56.654 INFO:teuthology.orchestra.run.smithi107.stdout:406 trunc from 0xcfd2800 to 0x9538600 2024-09-11T09:28:56.689 INFO:teuthology.orchestra.run.smithi107.stdout:407 read 0x85b9c00 thru 0x85c7894 (0xdc95 bytes) 2024-09-11T09:28:56.698 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:56.699+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:56.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:28:56.709 INFO:teuthology.orchestra.run.smithi107.stdout:410 write 0xc525000 thru 0xc52ef7c (0x9f7d bytes) 2024-09-11T09:28:56.718 INFO:teuthology.orchestra.run.smithi107.stdout:411 trunc from 0xc52f000 to 0x52f6800 2024-09-11T09:28:56.728 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:28:56.728 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:28:56.728 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-11T09:28:56.728 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T09:28:56.728 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T09:28:56.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-11T09:28:56.752 INFO:teuthology.orchestra.run.smithi107.stdout:413 write 0x1a34c00 thru 0x1a3ba31 (0x6e32 bytes) 2024-09-11T09:28:56.756 INFO:teuthology.orchestra.run.smithi107.stdout:414 trunc from 0x52f6800 to 0x7c06c00 2024-09-11T09:28:56.797 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:56.797 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:56.797 INFO:teuthology.orchestra.run.smithi107.stdout:417 write 0xd9b9e00 thru 0xd9bc28d (0x248e bytes) 2024-09-11T09:28:56.798 INFO:teuthology.orchestra.run.smithi107.stdout:419 write 0x2753000 thru 0x275d9bd (0xa9be bytes) 2024-09-11T09:28:56.798 INFO:teuthology.orchestra.run.smithi107.stdout:421 punch from 0x947ec00 to 0x9485400, (0x6800 bytes) 2024-09-11T09:28:56.799 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:56.799+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:56.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:28:56.807 INFO:teuthology.orchestra.run.smithi107.stdout:422 trunc from 0xd9bc400 to 0x6b14400 2024-09-11T09:28:56.827 INFO:teuthology.orchestra.run.smithi107.stdout:423 punch from 0x22be000 to 0x22c5200, (0x7200 bytes) 2024-09-11T09:28:56.827 INFO:teuthology.orchestra.run.smithi107.stdout:424 read 0x5484000 thru 0x548eef9 (0xaefa bytes) 2024-09-11T09:28:56.827 INFO:teuthology.orchestra.run.smithi107.stdout:425 read 0x3618e00 thru 0x36277f5 (0xe9f6 bytes) 2024-09-11T09:28:56.835 INFO:teuthology.orchestra.run.smithi107.stdout:428 trunc from 0x6b14400 to 0x82d600 2024-09-11T09:28:56.853 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:56.853 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T09:28:56.853 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:56.863 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:56.863 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:56.882 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:56.882 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:56.900 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:56.900 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:56.900 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:56.900+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:56.906 INFO:teuthology.orchestra.run.smithi107.stdout:432 write 0xc46c000 thru 0xc477b47 (0xbb48 bytes) 2024-09-11T09:28:56.917 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:56.918 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:56.924 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:56.924 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:56.924 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:433 write 0xc90a400 thru 0xc91a301 (0xff02 bytes) 2024-09-11T09:28:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:435 write 0x74b0c00 thru 0x74b8bbf (0x7fc0 bytes) 2024-09-11T09:28:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:437 write 0x3337e00 thru 0x333a6d7 (0x28d8 bytes) 2024-09-11T09:28:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:438 read 0x4406e00 thru 0x44145d7 (0xd7d8 bytes) 2024-09-11T09:28:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:440 write 0xb1be600 thru 0xb1c9357 (0xad58 bytes) 2024-09-11T09:28:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:442 read 0x2373400 thru 0x23777c6 (0x43c7 bytes) 2024-09-11T09:28:56.927 INFO:teuthology.orchestra.run.smithi107.stdout:444 read 0x3f50600 thru 0x3f5905d (0x8a5e bytes) 2024-09-11T09:28:56.927 INFO:teuthology.orchestra.run.smithi107.stdout:445 write 0x1dd6400 thru 0x1ddc013 (0x5c14 bytes) 2024-09-11T09:28:56.927 INFO:teuthology.orchestra.run.smithi107.stdout:448 punch from 0xab7f400 to 0xab83e00, (0x4a00 bytes) 2024-09-11T09:28:56.927 INFO:teuthology.orchestra.run.smithi107.stdout:449 write 0x6903e00 thru 0x6910f7f (0xd180 bytes) 2024-09-11T09:28:56.927 INFO:teuthology.orchestra.run.smithi107.stdout:451 read 0x5e52000 thru 0x5e5b9ac (0x99ad bytes) 2024-09-11T09:28:56.927 INFO:teuthology.orchestra.run.smithi107.stdout:452 read 0xbf63e00 thru 0xbf72899 (0xea9a bytes) 2024-09-11T09:28:56.934 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:28:56.935 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:56.936 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:56.940 INFO:teuthology.orchestra.run.smithi107.stdout:453 trunc from 0xc91a400 to 0x19a0200 2024-09-11T09:28:56.956 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:56.956 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:56.979 INFO:teuthology.orchestra.run.smithi107.stdout:454 read 0x1493600 thru 0x14988c4 (0x52c5 bytes) 2024-09-11T09:28:56.996 INFO:teuthology.orchestra.run.smithi107.stdout:455 trunc from 0x19a0200 to 0xde2fa00 2024-09-11T09:28:57.000 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:57.001+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:57.014 INFO:teuthology.orchestra.run.smithi107.stdout:456 write 0x5c6f600 thru 0x5c7a3f4 (0xadf5 bytes) 2024-09-11T09:28:57.014 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xec59800 2024-09-11T09:28:57.015 INFO:teuthology.orchestra.run.smithi107.stdout:458 trunc from 0xde2fa00 to 0xec59800 2024-09-11T09:28:57.036 INFO:teuthology.orchestra.run.smithi107.stdout:460 write 0xae71c00 thru 0xae7ceb2 (0xb2b3 bytes) 2024-09-11T09:28:57.046 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:57.046 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:57.047 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:57.047 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:57.052 INFO:teuthology.orchestra.run.smithi107.stdout:462 trunc from 0xec59800 to 0x1a9ea00 2024-09-11T09:28:57.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:28:57.093 INFO:teuthology.orchestra.run.smithi107.stdout:465 write 0x2705600 thru 0x270a4eb (0x4eec bytes) 2024-09-11T09:28:57.093 INFO:teuthology.orchestra.run.smithi107.stdout:468 read 0x129ee00 thru 0x12a7b5d (0x8d5e bytes) 2024-09-11T09:28:57.094 INFO:teuthology.orchestra.run.smithi107.stdout:470 read 0x3aba00 thru 0x3b165b (0x5c5c bytes) 2024-09-11T09:28:57.101 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:57.102+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:57.112 INFO:teuthology.orchestra.run.smithi107.stdout:472 write 0x3428800 thru 0x342d0d8 (0x48d9 bytes) 2024-09-11T09:28:57.113 INFO:teuthology.orchestra.run.smithi107.stdout:474 read 0x2e0c800 thru 0x2e0f389 (0x2b8a bytes) 2024-09-11T09:28:57.113 INFO:teuthology.orchestra.run.smithi107.stdout:476 read 0x160f800 thru 0x161e16f (0xe970 bytes) 2024-09-11T09:28:57.144 INFO:teuthology.orchestra.run.smithi107.stdout:477 write 0xe2bec00 thru 0xe2c9015 (0xa416 bytes) 2024-09-11T09:28:57.147 INFO:teuthology.orchestra.run.smithi107.stdout:482 trunc from 0xe2c9200 to 0xb65f600 2024-09-11T09:28:57.165 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:337 3.0 3:01c0f1ae:::rbd_data.11064c3f4ab3.0000000000000000:4 [read 13860864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570367+0000", 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5948536739999999, 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.010820738, 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 337 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570367+0000", 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570367+0000", 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570367+0000", 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570369+0000", 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1710000000000001e-06 2024-09-11T09:28:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570376+0000", 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4819999999999999e-06 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570381+0000", 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8330000000000001e-06 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.580977+0000", 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01059627 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581071+0000", 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3605999999999998e-05 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581187+0000", 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011695 2024-09-11T09:28:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:338 3.0 3:01c0f1ae:::rbd_data.11064c3f4ab3.0000000000000000:4 [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570388+0000", 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5948323150000001, 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011040705, 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 338 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570388+0000", 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570388+0000", 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570389+0000", 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.32e-07 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570391+0000", 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.227e-06 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570397+0000", 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7039999999999997e-06 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570406+0000", 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6479999999999992e-06 2024-09-11T09:28:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581231+0000", 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01082485 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581322+0000", 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1198000000000003e-05 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581429+0000", 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000106646 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:339 3.0 3:01c0f1ae:::rbd_data.11064c3f4ab3.0000000000000000:4 [read 14385152~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570414+0000", 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5948069079999998, 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011161740999999999, 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 339 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570414+0000", 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570414+0000", 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570414+0000", 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570416+0000", 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570423+0000", 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8369999999999999e-06 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570428+0000", 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1019999999999996e-06 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581466+0000", 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011037475999999999 2024-09-11T09:28:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581512+0000", 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6569000000000001e-05 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581575+0000", 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3033000000000004e-05 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:340 3.0 3:01c0f1ae:::rbd_data.11064c3f4ab3.0000000000000000:4 [read 14647296~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570435+0000", 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5947854079999999, 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011368484, 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 340 2024-09-11T09:28:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570435+0000", 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570435+0000", 2024-09-11T09:28:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570435+0000", 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0499999999999999e-07 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570438+0000", 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1449999999999998e-06 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570444+0000", 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570449+0000", 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.434e-06 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581619+0000", 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011170045999999999 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581700+0000", 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0391999999999999e-05 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581803+0000", 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103663 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:369 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 12288000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570726+0000", 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5944943999999999, 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011041502999999999, 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 369 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570726+0000", 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570726+0000", 2024-09-11T09:28:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570726+0000", 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5200000000000002e-07 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570729+0000", 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0779999999999998e-06 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570735+0000", 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570740+0000", 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581607+0000", 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.010866542999999999 2024-09-11T09:28:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581674+0000", 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6958999999999994e-05 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581768+0000", 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3999999999999994e-05 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:370 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 12550144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570747+0000", 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5944734999999999, 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011191069999999999, 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 370 2024-09-11T09:28:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570747+0000", 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570747+0000", 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570748+0000", 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.99e-07 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570750+0000", 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8210000000000002e-06 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570757+0000", 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4230000000000002e-06 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570762+0000", 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8620000000000002e-06 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581796+0000", 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011034571999999999 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581878+0000", 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1531000000000003e-05 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581938+0000", 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0262e-05 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:371 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 12812288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570768+0000", 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5944521709999999, 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011434069999999999, 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 371 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570768+0000", 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570768+0000", 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570769+0000", 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3599999999999999e-07 2024-09-11T09:28:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570771+0000", 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.086e-06 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570777+0000", 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2319999999999998e-06 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570782+0000", 2024-09-11T09:28:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1499999999999998e-06 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.581988+0000", 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011206145000000001 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582090+0000", 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101906 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582202+0000", 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011211500000000001 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:372 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 13074432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570789+0000", 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5944316189999999, 2024-09-11T09:28:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011675998999999999, 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 372 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570789+0000", 2024-09-11T09:28:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570789+0000", 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570789+0000", 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3799999999999998e-07 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570791+0000", 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.074e-06 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570799+0000", 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9319999999999996e-06 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570804+0000", 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0130000000000001e-06 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582251+0000", 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011446998999999999 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582350+0000", 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8435000000000002e-05 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582465+0000", 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000115108 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:373 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 13336576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570811+0000", 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.594409626, 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011907859999999999, 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 373 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570811+0000", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570811+0000", 2024-09-11T09:28:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570811+0000", 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999997e-07 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570814+0000", 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.21e-06 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570820+0000", 2024-09-11T09:28:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2110000000000003e-06 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570825+0000", 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9629999999999997e-06 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582508+0000", 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011683548 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582617+0000", 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0001091 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582719+0000", 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101314 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:374 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570832+0000", 2024-09-11T09:28:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5943879509999999, 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012147931000000001, 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 374 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570832+0000", 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570832+0000", 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570833+0000", 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570835+0000", 2024-09-11T09:28:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1150000000000001e-06 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570842+0000", 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5529999999999998e-06 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570847+0000", 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6720000000000001e-06 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582768+0000", 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.011920415 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582869+0000", 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101223 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.582980+0000", 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011147899999999999 2024-09-11T09:28:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:375 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 13860864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570862+0000", 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.594358406, 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012361382000000001, 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 375 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570862+0000", 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570862+0000", 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570863+0000", 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.37e-07 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570865+0000", 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3800000000000001e-06 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570872+0000", 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4000000000000003e-06 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570877+0000", 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583022+0000", 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012144392 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583122+0000", 2024-09-11T09:28:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9773999999999997e-05 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583223+0000", 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101771 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:376 3.8 3:14acc3da:::rbd_data.11064c3f4ab3.0000000000000004:4 [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570885+0000", 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5943352690000001, 2024-09-11T09:28:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012592192, 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 376 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570885+0000", 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570885+0000", 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570886+0000", 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8800000000000003e-07 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570888+0000", 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1749999999999999e-06 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570894+0000", 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3500000000000002e-06 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570900+0000", 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4960000000000004e-06 2024-09-11T09:28:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583271+0000", 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012371160000000001 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583367+0000", 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6229000000000003e-05 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583477+0000", 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000110294 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:377 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 12288000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570907+0000", 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5943138110000001, 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012808187, 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 377 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570907+0000", 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570907+0000", 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570907+0000", 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9900000000000001e-07 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570909+0000", 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0719999999999998e-06 2024-09-11T09:28:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570915+0000", 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2230000000000004e-06 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570920+0000", 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583517+0000", 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012597066000000001 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583607+0000", 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.967e-05 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583715+0000", 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010764099999999999 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:378 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 12550144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570928+0000", 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5942927099999999, 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01304992, 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 378 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570928+0000", 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570928+0000", 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570928+0000", 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0299999999999999e-07 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570936+0000", 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2400000000000007e-06 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570944+0000", 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4270000000000002e-06 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570949+0000", 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3519999999999997e-06 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583764+0000", 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.012814526999999999 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583868+0000", 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010442000000000001 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.583978+0000", 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010945100000000001 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:379 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 12812288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570957+0000", 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5942636390000002, 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013277924999999999, 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 379 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570957+0000", 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570957+0000", 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570957+0000", 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4200000000000001e-07 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570959+0000", 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1090000000000001e-06 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570966+0000", 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.365e-06 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570971+0000", 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9309999999999999e-06 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584020+0000", 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013049194 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584115+0000", 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5119999999999997e-05 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584235+0000", 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000119764 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:380 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 13074432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570977+0000", 2024-09-11T09:28:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5942432479999997, 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013514567, 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 380 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570977+0000", 2024-09-11T09:28:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570977+0000", 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570978+0000", 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9999999999999998e-07 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570980+0000", 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570988+0000", 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5120000000000008e-06 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570993+0000", 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.758e-06 2024-09-11T09:28:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584283+0000", 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013290244 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584380+0000", 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6768000000000006e-05 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584492+0000", 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111759 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:381 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 13336576~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.570999+0000", 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.594221042, 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013740629000000001, 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 381 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570999+0000", 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.570999+0000", 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571000+0000", 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6100000000000001e-07 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571003+0000", 2024-09-11T09:28:57.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7920000000000001e-06 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571009+0000", 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3949999999999998e-06 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571014+0000", 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9919999999999998e-06 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584533+0000", 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013519296 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584635+0000", 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010166300000000001 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584740+0000", 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010493 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.245 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:382 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5941988079999998, 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013989850999999999, 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 382 2024-09-11T09:28:57.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2e-07 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571024+0000", 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3209999999999999e-06 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571031+0000", 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571036+0000", 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1749999999999996e-06 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584791+0000", 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013754377 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584896+0000", 2024-09-11T09:28:57.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105188 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585011+0000", 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000115741 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:383 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 13860864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.5941777610000001, 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014215478, 2024-09-11T09:28:57.250 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 383 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571053+0000", 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4579999999999999e-06 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571060+0000", 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9829999999999999e-06 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571066+0000", 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1240000000000002e-06 2024-09-11T09:28:57.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585059+0000", 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013993798 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585157+0000", 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7528999999999994e-05 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585258+0000", 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101104 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:384 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.571074+0000", 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.594146383, 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014425392, 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.255 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 384 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571074+0000", 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571074+0000", 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571075+0000", 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8400000000000005e-07 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571077+0000", 2024-09-11T09:28:57.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1600000000000001e-06 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571083+0000", 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3300000000000004e-06 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571088+0000", 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.78e-06 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585291+0000", 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014203386 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585386+0000", 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4495999999999998e-05 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585499+0000", 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113756 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.260 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.260 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:57.261 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:57.261+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:483 trunc from 0xb65f600 to 0x24ca200 2024-09-11T09:28:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:484 trunc from 0x24ca200 to 0xe4df400 2024-09-11T09:28:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:485 trunc from 0xe4df400 to 0xea94800 2024-09-11T09:28:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:486 write 0xbafa00 thru 0xbb5b18 (0x6119 bytes) 2024-09-11T09:28:57.262 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:28:57.265 INFO:teuthology.orchestra.run.smithi107.stdout:487 trunc from 0xea94800 to 0x747fa00 2024-09-11T09:28:57.304 INFO:teuthology.orchestra.run.smithi107.stdout:488 write 0xbbeba00 thru 0xbbf326a (0x786b bytes) 2024-09-11T09:28:57.304 INFO:teuthology.orchestra.run.smithi107.stdout:490 punch from 0xa65dc00 to 0xa663e00, (0x6200 bytes) 2024-09-11T09:28:57.327 INFO:teuthology.orchestra.run.smithi107.stdout:493 write 0xce66800 thru 0xce736f7 (0xcef8 bytes) 2024-09-11T09:28:57.327 INFO:teuthology.orchestra.run.smithi107.stdout:494 punch from 0x72f7c00 to 0x72ff000, (0x7400 bytes) 2024-09-11T09:28:57.327 INFO:teuthology.orchestra.run.smithi107.stdout:495 read 0x2e73600 thru 0x2e7dca6 (0xa6a7 bytes) 2024-09-11T09:28:57.349 INFO:teuthology.orchestra.run.smithi107.stdout:496 write 0xd634600 thru 0xd63eaf7 (0xa4f8 bytes) 2024-09-11T09:28:57.349 INFO:teuthology.orchestra.run.smithi107.stdout:498 read 0x3955800 thru 0x3962675 (0xce76 bytes) 2024-09-11T09:28:57.358 INFO:teuthology.orchestra.run.smithi107.stdout:499 trunc from 0xd63ec00 to 0x54d8200 2024-09-11T09:28:57.361 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:57.362+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:57.384 INFO:teuthology.orchestra.run.smithi107.stdout:508 punch from 0x1171600 to 0x1171a00, (0x400 bytes) 2024-09-11T09:28:57.404 INFO:teuthology.orchestra.run.smithi107.stdout:509 write 0x9970000 thru 0x997249b (0x249c bytes) 2024-09-11T09:28:57.414 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:57.415 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:57.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:28:57.415 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:57.424 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:510 write 0xec85200 thru 0xec8793c (0x273d bytes) 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:511 write 0xa2f3000 thru 0xa2ffa8d (0xca8e bytes) 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:512 write 0x806e800 thru 0x807b398 (0xcb99 bytes) 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:513 write 0x7eb1600 thru 0x7eb20bc (0xabd bytes) 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:515 write 0x7528200 thru 0x7531cb9 (0x9aba bytes) 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:517 punch from 0x662ee00 to 0x6632e00, (0x4000 bytes) 2024-09-11T09:28:57.428 INFO:teuthology.orchestra.run.smithi107.stdout:519 trunc from 0xec87a00 to 0xe5f9200 2024-09-11T09:28:57.463 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:57.463+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:57.463 INFO:teuthology.orchestra.run.smithi107.stdout:521 punch from 0x43a3a00 to 0x43b3000, (0xf600 bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:522 read 0x5bbb600 thru 0x5bbb88c (0x28d bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:523 write 0x6278400 thru 0x627de14 (0x5a15 bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:525 write 0x55d7c00 thru 0x55e5a31 (0xde32 bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:527 write 0x4df7400 thru 0x4e0109e (0x9c9f bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:530 read 0x10dc600 thru 0x10e9aa6 (0xd4a7 bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:532 write 0x14e3800 thru 0x14f2e32 (0xf633 bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:535 read 0x8365400 thru 0x836e3a5 (0x8fa6 bytes) 2024-09-11T09:28:57.464 INFO:teuthology.orchestra.run.smithi107.stdout:536 write 0x8e4800 thru 0x8e7b96 (0x3397 bytes) 2024-09-11T09:28:57.465 INFO:teuthology.orchestra.run.smithi107.stdout:538 punch from 0xcc35200 to 0xcc3da00, (0x8800 bytes) 2024-09-11T09:28:57.465 INFO:teuthology.orchestra.run.smithi107.stdout:539 read 0xb3cd800 thru 0xb3ce8de (0x10df bytes) 2024-09-11T09:28:57.482 INFO:teuthology.orchestra.run.smithi107.stdout:541 trunc from 0xe5f9200 to 0x743200 2024-09-11T09:28:57.526 INFO:teuthology.orchestra.run.smithi107.stdout:543 trunc from 0x743200 to 0x1300c00 2024-09-11T09:28:57.538 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:28:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:28:57.538 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:28:57.538 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:57.547 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:28:57.554 INFO:teuthology.orchestra.run.smithi107.stdout:544 write 0x1dbd800 thru 0x1dc1b8f (0x4390 bytes) 2024-09-11T09:28:57.555 INFO:teuthology.orchestra.run.smithi107.stdout:545 flatten 2024-09-11T09:28:57.590 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:57.591+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:57.592 INFO:teuthology.orchestra.run.smithi107.stdout:546 write 0x34c9000 thru 0x34cc80e (0x380f bytes) 2024-09-11T09:28:57.593 INFO:teuthology.orchestra.run.smithi107.stdout:547 read 0x249ae00 thru 0x24aaacc (0xfccd bytes) 2024-09-11T09:28:57.593 INFO:teuthology.orchestra.run.smithi107.stdout:549 read 0x2e30c00 thru 0x2e3b5d7 (0xa9d8 bytes) 2024-09-11T09:28:57.593 INFO:teuthology.orchestra.run.smithi107.stdout:551 read 0x8b600 thru 0x93106 (0x7b07 bytes) 2024-09-11T09:28:57.615 INFO:teuthology.orchestra.run.smithi107.stdout:553 write 0xb067a00 thru 0xb06a3e7 (0x29e8 bytes) 2024-09-11T09:28:57.635 INFO:teuthology.orchestra.run.smithi107.stdout:555 write 0xb585c00 thru 0xb5924c8 (0xc8c9 bytes) 2024-09-11T09:28:57.641 INFO:teuthology.orchestra.run.smithi107.stdout:556 clone 4 order 21 su 16384 sc 9 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.198997058, 2024-09-11T09:28:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:28:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:28:57.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.9374773699999999, 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.663 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:28:57.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:28:57.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:57.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:28:57.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.9362314030000007, 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:28:57.670 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:28:57.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:28:57.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:28:57.674 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:28:57.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:28:57.676 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:57.676+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:57.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:28:57.781 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:28:57.782+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:28:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:57.787 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:57.795 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:28:57.878 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:57.878+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:57.901 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:57.901 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:57.901 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:57.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:28:57.979 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:28:57.980+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:28:58.059 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:58.059 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:28:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:28:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:28:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.060 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.620892132000002, 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:28:58.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:28:58.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036357786, 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059776040000000001, 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 111 2024-09-11T09:28:58.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021534+0000", 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.23e-07 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021536+0000", 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0449999999999999e-06 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021540+0000", 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.596e-06 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021542+0000", 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6510000000000001e-06 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025459+0000", 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0039162329999999999 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025518+0000", 2024-09-11T09:28:58.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9490999999999997e-05 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025591+0000", 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2967999999999997e-05 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027405+0000", 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001813989 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027421+0000", 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5807999999999998e-05 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027471+0000", 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0529999999999999e-05 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027511+0000", 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0070000000000001e-05 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036332255000001, 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006066391, 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.077 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 112 2024-09-11T09:28:58.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021559+0000", 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021560+0000", 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9999999999999999e-07 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021620+0000", 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0795000000000003e-05 2024-09-11T09:28:58.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021637+0000", 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6197000000000001e-05 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021643+0000", 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4640000000000003e-06 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025831+0000", 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0041879650000000001 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025882+0000", 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0767999999999999e-05 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.025956+0000", 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4639000000000001e-05 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027532+0000", 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001575885 2024-09-11T09:28:58.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027563+0000", 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0768000000000001e-05 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027602+0000", 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8522000000000001e-05 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027626+0000", 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4088e-05 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036237524000001, 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061017809999999997, 2024-09-11T09:28:58.085 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 113 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021654+0000", 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021655+0000", 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2600000000000002e-07 2024-09-11T09:28:58.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021659+0000", 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9920000000000001e-06 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021675+0000", 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6217000000000001e-05 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021681+0000", 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5840000000000002e-06 2024-09-11T09:28:58.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026482+0000", 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0048014049999999999 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026534+0000", 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2002000000000001e-05 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.026607+0000", 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2936999999999996e-05 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027676+0000", 2024-09-11T09:28:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010690739999999999 2024-09-11T09:28:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027687+0000", 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.131e-05 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027734+0000", 2024-09-11T09:28:58.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6323999999999999e-05 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027756+0000", 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2310000000000002e-05 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.095 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036180941000001, 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071833610000000001, 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 114 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021711+0000", 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-07 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021714+0000", 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1609999999999999e-06 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021719+0000", 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.04e-06 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021722+0000", 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9129999999999999e-06 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027008+0000", 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052859259999999998 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027057+0000", 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9332000000000001e-05 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027135+0000", 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7935999999999994e-05 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028758+0000", 2024-09-11T09:28:58.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016231100000000001 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028793+0000", 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4656000000000002e-05 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028846+0000", 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2459e-05 2024-09-11T09:28:58.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028894+0000", 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8458999999999998e-05 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036161806999999, 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0076141910000000002, 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 115 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021730+0000", 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8099999999999999e-07 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021777+0000", 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7073999999999997e-05 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021782+0000", 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3360000000000002e-06 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021786+0000", 2024-09-11T09:28:58.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2289999999999999e-06 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027795+0000", 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0060089870000000004 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027876+0000", 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1749999999999995e-05 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.027998+0000", 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121305 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029211+0000", 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001213638 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029256+0000", 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4236999999999997e-05 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029307+0000", 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0927999999999998e-05 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029344+0000", 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7425999999999997e-05 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.108 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036100439999998, 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0083223069999999993, 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 116 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021791+0000", 2024-09-11T09:28:58.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021794+0000", 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021798+0000", 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.551e-06 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021800+0000", 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6000000000000001e-06 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028416+0000", 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0066157569999999999 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028464+0000", 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8229999999999997e-05 2024-09-11T09:28:58.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028537+0000", 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2516999999999997e-05 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029379+0000", 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00084212600000000001 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030018+0000", 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000638909 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030072+0000", 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4292000000000001e-05 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030113+0000", 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1335999999999999e-05 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.036087669, 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0085277849999999995, 2024-09-11T09:28:58.115 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 117 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021804+0000", 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2700000000000001e-07 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021806+0000", 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.199e-06 2024-09-11T09:28:58.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021810+0000", 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7759999999999999e-06 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.021813+0000", 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6900000000000001e-06 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028937+0000", 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0071237260000000004 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.028983+0000", 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6090000000000001e-05 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029057+0000", 2024-09-11T09:28:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4425000000000007e-05 2024-09-11T09:28:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.029591+0000", 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00053415799999999996 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030264+0000", 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067266199999999998 2024-09-11T09:28:58.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030294+0000", 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9635999999999999e-05 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.030332+0000", 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8195999999999998e-05 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.633521136999999, 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006285049, 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 137 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424370+0000", 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424371+0000", 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5199999999999998e-07 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424412+0000", 2024-09-11T09:28:58.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0991000000000001e-05 2024-09-11T09:28:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424419+0000", 2024-09-11T09:28:58.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6469999999999998e-06 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424427+0000", 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7479999999999993e-06 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428503+0000", 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0040758069999999999 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428583+0000", 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0288000000000001e-05 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.428703+0000", 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000119339 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430520+0000", 2024-09-11T09:28:58.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018175999999999999 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430568+0000", 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7411999999999999e-05 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430618+0000", 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0510000000000003e-05 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430656+0000", 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7455e-05 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.633455714, 2024-09-11T09:28:58.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0072699360000000003, 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 138 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424436+0000", 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424440+0000", 2024-09-11T09:28:58.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5300000000000001e-06 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424450+0000", 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9499999999999996e-06 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424454+0000", 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3669999999999999e-06 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429660+0000", 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052059150000000002 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429734+0000", 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3676000000000004e-05 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.429848+0000", 2024-09-11T09:28:58.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011422200000000001 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431304+0000", 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001455944 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431655+0000", 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00035130099999999999 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431683+0000", 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8172000000000001e-05 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431706+0000", 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2531000000000002e-05 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:58.138 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.633432306, 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0073326670000000002, 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 139 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424459+0000", 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424460+0000", 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4999999999999999e-07 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424462+0000", 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.942e-06 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424467+0000", 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2910000000000001e-06 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424469+0000", 2024-09-11T09:28:58.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.678e-06 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430155+0000", 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056853709999999998 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430201+0000", 2024-09-11T09:28:58.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6239000000000002e-05 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430288+0000", 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6768999999999995e-05 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431736+0000", 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001448048 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431748+0000", 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1824000000000001e-05 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431773+0000", 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5000000000000001e-05 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431792+0000", 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9255000000000001e-05 2024-09-11T09:28:58.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.633418584000001, 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0079260519999999994, 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 140 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424473+0000", 2024-09-11T09:28:58.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424517+0000", 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3239999999999999e-05 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424523+0000", 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424526+0000", 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.551e-06 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430683+0000", 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061561250000000001 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430725+0000", 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2803999999999999e-05 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.430793+0000", 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7183999999999999e-05 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431886+0000", 2024-09-11T09:28:58.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0010930270000000001 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432339+0000", 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045296099999999999 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432364+0000", 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5610999999999999e-05 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432399+0000", 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4919999999999998e-05 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.633356884000001, 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081622139999999992, 2024-09-11T09:28:58.152 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 141 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424535+0000", 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3600000000000003e-07 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424540+0000", 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8010000000000003e-06 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424546+0000", 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7679999999999997e-06 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.424551+0000", 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431331+0000", 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0067806139999999999 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431371+0000", 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0110000000000001e-05 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.431436+0000", 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5040000000000001e-05 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432172+0000", 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00073592299999999998 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432633+0000", 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000460247 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432658+0000", 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5282e-05 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:31.432697+0000", 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9069999999999997e-05 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4519.0:2303 3.16 e29/21 3:6a621994:::rbd_object_map.11062c268ffd:head v 29'188, mlcod=29'179)", 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:56.938491+0000", 2024-09-11T09:28:58.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.1194003079999999, 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005748469, 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938491+0000", 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938491+0000", 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938492+0000", 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938496+0000", 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4870000000000004e-06 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938505+0000", 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9209999999999998e-06 2024-09-11T09:28:58.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938510+0000", 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5349999999999998e-06 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938550+0000", 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9925000000000001e-05 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.938564+0000", 2024-09-11T09:28:58.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.385e-05 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.944202+0000", 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056381859999999999 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.944240+0000", 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7913000000000001e-05 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4519.0:2389 3.a 3:573d1aec:::rbd_data.11062c268ffd.00000000000000c3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:56.955397+0000", 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.1024948939999999, 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056827140000000002, 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4519", 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1568279568", 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2389 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.163 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955397+0000", 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955397+0000", 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955397+0000", 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7400000000000002e-07 2024-09-11T09:28:58.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955401+0000", 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8469999999999998e-06 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955411+0000", 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.005e-05 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955419+0000", 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.306e-06 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955468+0000", 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9311000000000003e-05 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955528+0000", 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9561000000000001e-05 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955627+0000", 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9071000000000001e-05 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.958292+0000", 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026652300000000002 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.960992+0000", 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027005079999999999 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.961041+0000", 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8470000000000002e-05 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.961079+0000", 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8586e-05 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4519.0:2392 3.14 3:2a598208:::rbd_data.11062c268ffd.00000000000000c4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:56.955536+0000", 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.1023559869999999, 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055632119999999997, 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4519", 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1568279568", 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2392 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955536+0000", 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955536+0000", 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955536+0000", 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6300000000000005e-07 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955540+0000", 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5540000000000002e-06 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955551+0000", 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1212999999999999e-05 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955558+0000", 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1940000000000001e-06 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955603+0000", 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5182999999999998e-05 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955676+0000", 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2644000000000004e-05 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.955774+0000", 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7884e-05 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.958489+0000", 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0027148850000000002 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.961031+0000", 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.002541939 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.961073+0000", 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2710000000000003e-05 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:56.961099+0000", 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5343000000000001e-05 2024-09-11T09:28:58.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4519.0:3569 3.16 e29/21 3:6a621994:::rbd_object_map.11062c268ffd:head v 29'309, mlcod=29'303)", 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:57.469039+0000", 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.58885283399999999, 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059158659999999997, 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469039+0000", 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469039+0000", 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469039+0000", 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.13e-07 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469044+0000", 2024-09-11T09:28:58.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9019999999999998e-06 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469056+0000", 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1591e-05 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469062+0000", 2024-09-11T09:28:58.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8170000000000004e-06 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469107+0000", 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4978999999999997e-05 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.469126+0000", 2024-09-11T09:28:58.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9258000000000001e-05 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474921+0000", 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057948260000000003 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474955+0000", 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3880000000000001e-05 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.180 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4519.0:3573 3.16 e29/21 3:6a621994:::rbd_object_map.11062c268ffd:head v 29'313, mlcod=29'304)", 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:57.471543+0000", 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.58634847300000004, 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0058969139999999996, 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.181 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471543+0000", 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471543+0000", 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471544+0000", 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9500000000000002e-07 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471549+0000", 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4450000000000004e-06 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471565+0000", 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.537e-05 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471571+0000", 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6239999999999996e-06 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471614+0000", 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2246000000000002e-05 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.471634+0000", 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0203e-05 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.477399+0000", 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0057655700000000002 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.477440+0000", 2024-09-11T09:28:58.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0760999999999998e-05 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(client.4519.0:3577 3.16 e29/21 3:6a621994:::rbd_object_map.11062c268ffd:head v 29'316, mlcod=29'307)", 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:57.474480+0000", 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.58341167199999999, 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0056543399999999999, 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474480+0000", 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474480+0000", 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474481+0000", 2024-09-11T09:28:58.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.99e-07 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474485+0000", 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8629999999999999e-06 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474495+0000", 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5580000000000006e-06 2024-09-11T09:28:58.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474501+0000", 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9429999999999997e-06 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474546+0000", 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5317e-05 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.474565+0000", 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8420999999999999e-05 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.480083+0000", 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0055180029999999996 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.480134+0000", 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1635999999999998e-05 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4519.0:3958 3.1e 3:7a4cddfe:::rbd_data.11062c268ffd.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:57.567471+0000", 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.490420825, 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006780973, 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4519", 2024-09-11T09:28:58.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1568279568", 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3958 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.567471+0000", 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.567471+0000", 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.567472+0000", 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2600000000000001e-07 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.569677+0000", 2024-09-11T09:28:58.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0022051940000000002 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.569686+0000", 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2450000000000004e-06 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.569691+0000", 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1270000000000002e-06 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.569748+0000", 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6527000000000003e-05 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.569833+0000", 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5402999999999995e-05 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.570037+0000", 2024-09-11T09:28:58.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000203523 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.571563+0000", 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001526283 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.574186+0000", 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0026231539999999999 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:28:58.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.574223+0000", 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6903000000000002e-05 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:57.574252+0000", 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8688e-05 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.197 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:58.198 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-11T09:28:58.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:28:58.199 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:58.199+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:58.251 INFO:teuthology.orchestra.run.smithi107.stdout:558 write 0xa7de000 thru 0xa7edd51 (0xfd52 bytes) 2024-09-11T09:28:58.252 INFO:teuthology.orchestra.run.smithi107.stdout:560 read 0x41d9e00 thru 0x41e589b (0xba9c bytes) 2024-09-11T09:28:58.286 INFO:teuthology.orchestra.run.smithi107.stdout:561 write 0xc85a000 thru 0xc8641f4 (0xa1f5 bytes) 2024-09-11T09:28:58.286 INFO:teuthology.orchestra.run.smithi107.stdout:563 punch from 0x362b200 to 0x3638400, (0xd200 bytes) 2024-09-11T09:28:58.300 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:58.300+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:58.316 INFO:teuthology.orchestra.run.smithi107.stdout:564 write 0xc9cba00 thru 0xc9d211a (0x671b bytes) 2024-09-11T09:28:58.316 INFO:teuthology.orchestra.run.smithi107.stdout:565 punch from 0x1d39000 to 0x1d3f000, (0x6000 bytes) 2024-09-11T09:28:58.316 INFO:teuthology.orchestra.run.smithi107.stdout:567 write 0x5cc6600 thru 0x5cc7072 (0xa73 bytes) 2024-09-11T09:28:58.316 INFO:teuthology.orchestra.run.smithi107.stdout:568 read 0x74d8000 thru 0x74e4787 (0xc788 bytes) 2024-09-11T09:28:58.316 INFO:teuthology.orchestra.run.smithi107.stdout:570 punch from 0x46e1000 to 0x46ef200, (0xe200 bytes) 2024-09-11T09:28:58.317 INFO:teuthology.orchestra.run.smithi107.stdout:571 write 0x85bdc00 thru 0x85be813 (0xc14 bytes) 2024-09-11T09:28:58.317 INFO:teuthology.orchestra.run.smithi107.stdout:572 write 0x1f0ac00 thru 0x1f151c1 (0xa5c2 bytes) 2024-09-11T09:28:58.330 INFO:teuthology.orchestra.run.smithi107.stdout:573 trunc from 0xc9d2200 to 0x62d800 2024-09-11T09:28:58.372 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:58.373 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:58.373 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:58.379 INFO:teuthology.orchestra.run.smithi107.stdout:574 read 0x5f1400 thru 0x5f71d8 (0x5dd9 bytes) 2024-09-11T09:28:58.381 INFO:teuthology.orchestra.run.smithi107.stdout:575 read 0x431200 thru 0x43f873 (0xe674 bytes) 2024-09-11T09:28:58.382 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:28:58.383 INFO:teuthology.orchestra.run.smithi107.stdout:577 punch from 0x2d2000 to 0x2d9a00, (0x7a00 bytes) 2024-09-11T09:28:58.396 INFO:teuthology.orchestra.run.smithi107.stdout:580 trunc from 0x62d800 to 0xcba1000 2024-09-11T09:28:58.408 INFO:teuthology.orchestra.run.smithi107.stdout:581 trunc from 0xcba1000 to 0xaab7600 2024-09-11T09:28:58.418 INFO:teuthology.orchestra.run.smithi107.stdout:582 punch from 0x998b000 to 0x998ca00, (0x1a00 bytes) 2024-09-11T09:28:58.425 INFO:teuthology.orchestra.run.smithi107.stdout:583 punch from 0x6f65e00 to 0x6f72a00, (0xcc00 bytes) 2024-09-11T09:28:58.425 INFO:teuthology.orchestra.run.smithi107.stdout:586 read 0x175000 thru 0x177f93 (0x2f94 bytes) 2024-09-11T09:28:58.425 INFO:teuthology.orchestra.run.smithi107.stdout:587 punch from 0x4ea2c00 to 0x4eb1a00, (0xee00 bytes) 2024-09-11T09:28:58.425 INFO:teuthology.orchestra.run.smithi107.stdout:589 write 0x525ee00 thru 0x526953c (0xa73d bytes) 2024-09-11T09:28:58.425 INFO:teuthology.orchestra.run.smithi107.stdout:590 trunc from 0xaab7600 to 0x9b5a200 2024-09-11T09:28:58.427 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:58.428+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:58.460 INFO:teuthology.orchestra.run.smithi107.stdout:595 read 0x51d9000 thru 0x51e7d58 (0xed59 bytes) 2024-09-11T09:28:58.461 INFO:teuthology.orchestra.run.smithi107.stdout:597 read 0x3920c00 thru 0x393026c (0xf66d bytes) 2024-09-11T09:28:58.461 INFO:teuthology.orchestra.run.smithi107.stdout:598 punch from 0x899a800 to 0x89a0200, (0x5a00 bytes) 2024-09-11T09:28:58.461 INFO:teuthology.orchestra.run.smithi107.stdout:599 write 0x76a0200 thru 0x76a7349 (0x714a bytes) 2024-09-11T09:28:58.462 INFO:teuthology.orchestra.run.smithi107.stdout:600 write 0x6200200 thru 0x620ef6c (0xed6d bytes) 2024-09-11T09:28:58.466 INFO:teuthology.orchestra.run.smithi107.stdout:601 trunc from 0x9b5a200 to 0x5a66a00 2024-09-11T09:28:58.501 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:58.502+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:58.505 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:58.505 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:58.506 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:58.506 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:58.506 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:58.508 INFO:teuthology.orchestra.run.smithi107.stdout:603 punch from 0x4e7ec00 to 0x4e82a00, (0x3e00 bytes) 2024-09-11T09:28:58.512 INFO:teuthology.orchestra.run.smithi107.stdout:605 read 0x2808c00 thru 0x280ebff (0x6000 bytes) 2024-09-11T09:28:58.512 INFO:teuthology.orchestra.run.smithi107.stdout:606 punch from 0x200a800 to 0x201a600, (0xfe00 bytes) 2024-09-11T09:28:58.512 INFO:teuthology.orchestra.run.smithi107.stdout:613 read 0x9bdc00 thru 0x9c3393 (0x5794 bytes) 2024-09-11T09:28:58.513 INFO:teuthology.orchestra.run.smithi107.stdout:614 punch from 0x25f3800 to 0x2602800, (0xf000 bytes) 2024-09-11T09:28:58.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:28:58.520 INFO:teuthology.orchestra.run.smithi107.stdout:616 trunc from 0x5a66a00 to 0xb9df600 2024-09-11T09:28:58.534 INFO:teuthology.orchestra.run.smithi107.stdout:617 read 0x8d5c000 thru 0x8d5d11d (0x111e bytes) 2024-09-11T09:28:58.542 INFO:teuthology.orchestra.run.smithi107.stdout:618 read 0x79fc400 thru 0x7a07b70 (0xb771 bytes) 2024-09-11T09:28:58.542 INFO:teuthology.orchestra.run.smithi107.stdout:620 read 0x7214400 thru 0x72169c6 (0x25c7 bytes) 2024-09-11T09:28:58.542 INFO:teuthology.orchestra.run.smithi107.stdout:621 read 0x68a7200 thru 0x68acd94 (0x5b95 bytes) 2024-09-11T09:28:58.552 INFO:teuthology.orchestra.run.smithi107.stdout:622 write 0xd2bc600 thru 0xd2c1b84 (0x5585 bytes) 2024-09-11T09:28:58.553 INFO:teuthology.orchestra.run.smithi107.stdout:623 read 0x83c6800 thru 0x83cb074 (0x4875 bytes) 2024-09-11T09:28:58.553 INFO:teuthology.orchestra.run.smithi107.stdout:624 write 0xb75ae00 thru 0xb76411c (0x931d bytes) 2024-09-11T09:28:58.553 INFO:teuthology.orchestra.run.smithi107.stdout:626 write 0x8ccde00 thru 0x8cd716c (0x936d bytes) 2024-09-11T09:28:58.555 INFO:teuthology.orchestra.run.smithi107.stdout:627 trunc from 0xd2c1c00 to 0xe453e00 2024-09-11T09:28:58.578 INFO:teuthology.orchestra.run.smithi107.stdout:628 trunc from 0xe453e00 to 0xad6fc00 2024-09-11T09:28:58.599 INFO:teuthology.orchestra.run.smithi107.stdout:631 read 0x8272000 thru 0x8273876 (0x1877 bytes) 2024-09-11T09:28:58.599 INFO:teuthology.orchestra.run.smithi107.stdout:632 read 0x2cd6c00 thru 0x2cdebeb (0x7fec bytes) 2024-09-11T09:28:58.600 INFO:teuthology.orchestra.run.smithi107.stdout:633 write 0xaad9800 thru 0xaae5e98 (0xc699 bytes) 2024-09-11T09:28:58.600 INFO:teuthology.orchestra.run.smithi107.stdout:634 read 0x3309a00 thru 0x3318668 (0xec69 bytes) 2024-09-11T09:28:58.600 INFO:teuthology.orchestra.run.smithi107.stdout:635 write 0x2f10e00 thru 0x2f144c8 (0x36c9 bytes) 2024-09-11T09:28:58.601 INFO:teuthology.orchestra.run.smithi107.stdout:636 punch from 0x9e3fe00 to 0x9e4ca00, (0xcc00 bytes) 2024-09-11T09:28:58.601 INFO:teuthology.orchestra.run.smithi107.stdout:637 write 0x442ba00 thru 0x442d831 (0x1e32 bytes) 2024-09-11T09:28:58.601 INFO:teuthology.orchestra.run.smithi107.stdout:639 write 0x8569000 thru 0x8574608 (0xb609 bytes) 2024-09-11T09:28:58.601 INFO:teuthology.orchestra.run.smithi107.stdout:640 punch from 0x1112c00 to 0x1118e00, (0x6200 bytes) 2024-09-11T09:28:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:643 punch from 0x7d11400 to 0x7d20e00, (0xfa00 bytes) 2024-09-11T09:28:58.602 INFO:teuthology.orchestra.run.smithi107.stdout:645 write 0x5188000 thru 0x5193ee1 (0xbee2 bytes) 2024-09-11T09:28:58.604 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:58.603+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:646 punch from 0x48b3800 to 0x48b9200, (0x5a00 bytes) 2024-09-11T09:28:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:647 read 0x8edb200 thru 0x8edc77f (0x1580 bytes) 2024-09-11T09:28:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:648 punch from 0x7f93800 to 0x7f97e00, (0x4600 bytes) 2024-09-11T09:28:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:649 read 0x4542200 thru 0x4545604 (0x3405 bytes) 2024-09-11T09:28:58.605 INFO:teuthology.orchestra.run.smithi107.stdout:650 write 0x4898600 thru 0x489aa2c (0x242d bytes) 2024-09-11T09:28:58.607 INFO:teuthology.orchestra.run.smithi107.stdout:651 trunc from 0xad6fc00 to 0x898f800 2024-09-11T09:28:58.672 INFO:teuthology.orchestra.run.smithi107.stdout:652 write 0xdc03e00 thru 0xdc1343b (0xf63c bytes) 2024-09-11T09:28:58.672 INFO:teuthology.orchestra.run.smithi107.stdout:653 write 0xac49800 thru 0xac52d41 (0x9542 bytes) 2024-09-11T09:28:58.673 INFO:teuthology.orchestra.run.smithi107.stdout:654 write 0xb55be00 thru 0xb56a46d (0xe66e bytes) 2024-09-11T09:28:58.673 INFO:teuthology.orchestra.run.smithi107.stdout:656 read 0x1797e00 thru 0x17a4140 (0xc341 bytes) 2024-09-11T09:28:58.673 INFO:teuthology.orchestra.run.smithi107.stdout:662 write 0xffde00 thru 0x100764c (0x984d bytes) 2024-09-11T09:28:58.673 INFO:teuthology.orchestra.run.smithi107.stdout:669 read 0x4fd5a00 thru 0x4fdce15 (0x7416 bytes) 2024-09-11T09:28:58.674 INFO:teuthology.orchestra.run.smithi107.stdout:670 write 0x9bfe200 thru 0x9c06082 (0x7e83 bytes) 2024-09-11T09:28:58.674 INFO:teuthology.orchestra.run.smithi107.stdout:674 write 0x345e000 thru 0x3465277 (0x7278 bytes) 2024-09-11T09:28:58.678 INFO:teuthology.orchestra.run.smithi107.stdout:677 trunc from 0xdc13600 to 0xa086a00 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:835 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 180224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33620156000000001, 2024-09-11T09:28:58.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012798782, 2024-09-11T09:28:58.691 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 835 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:28:58.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5699999999999998e-07 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352433+0000", 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9740000000000001e-06 2024-09-11T09:28:58.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352439+0000", 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7660000000000004e-06 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352443+0000", 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1910000000000002e-06 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365065+0000", 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012622517 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365134+0000", 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8604999999999998e-05 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365229+0000", 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5371999999999994e-05 2024-09-11T09:28:58.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:836 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.336183701, 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012941388, 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 836 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352451+0000", 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.832e-06 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352456+0000", 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3380000000000004e-06 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352461+0000", 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6319999999999999e-06 2024-09-11T09:28:58.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365259+0000", 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012797894000000001 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365327+0000", 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8069000000000002e-05 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365390+0000", 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3246999999999999e-05 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:837 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352466+0000", 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33616581499999998, 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013107196999999999, 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 837 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352466+0000", 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352466+0000", 2024-09-11T09:28:58.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352467+0000", 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.84e-07 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352469+0000", 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9860000000000001e-06 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352474+0000", 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3519999999999997e-06 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352479+0000", 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.459e-06 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365420+0000", 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012941104 2024-09-11T09:28:58.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365486+0000", 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6082999999999997e-05 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365574+0000", 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7829000000000002e-05 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:849 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 147456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352484+0000", 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33614792100000002, 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013236056, 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 849 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352484+0000", 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352484+0000", 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352485+0000", 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352490+0000", 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6670000000000001e-06 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352496+0000", 2024-09-11T09:28:58.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9869999999999996e-06 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352501+0000", 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3800000000000004e-06 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365604+0000", 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013103571 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365666+0000", 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1983999999999995e-05 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365720+0000", 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4103e-05 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:850 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 442368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33610767400000002, 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013366031, 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 850 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5000000000000003e-07 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352527+0000", 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1710000000000001e-06 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352534+0000", 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5050000000000004e-06 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352538+0000", 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6550000000000003e-06 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365748+0000", 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013209594 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365797+0000", 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9409000000000001e-05 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365891+0000", 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3146999999999994e-05 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:851 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 737280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352545+0000", 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33608722299999999, 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013868831, 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 851 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352545+0000", 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352545+0000", 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352546+0000", 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:28:58.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352547+0000", 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352553+0000", 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9229999999999999e-06 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352558+0000", 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365919+0000", 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013361708999999999 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365985+0000", 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5562999999999996e-05 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366414+0000", 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042899000000000002 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:852 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33606265800000001, 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013986192999999999, 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 852 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352744+0000", 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017385299999999999 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352754+0000", 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0334e-05 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352761+0000", 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6279999999999999e-06 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366442+0000", 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013681415000000001 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366493+0000", 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0263000000000003e-05 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366556+0000", 2024-09-11T09:28:58.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3176000000000006e-05 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:872 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 212992~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33537933599999997, 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013469549000000001, 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 872 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353255+0000", 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9929999999999998e-06 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353261+0000", 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0639999999999999e-06 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353266+0000", 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366589+0000", 2024-09-11T09:28:58.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013323536 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366651+0000", 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1544999999999994e-05 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366722+0000", 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1459999999999997e-05 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.731 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:873 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33536063100000002, 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013566754, 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 873 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353274+0000", 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353280+0000", 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0329999999999997e-06 2024-09-11T09:28:58.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353285+0000", 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5940000000000004e-06 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366748+0000", 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01346346 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366786+0000", 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7812999999999999e-05 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366838+0000", 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:874 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.736 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33534235299999998, 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01374672, 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 874 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2399999999999999e-07 2024-09-11T09:28:58.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353292+0000", 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353298+0000", 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353312+0000", 2024-09-11T09:28:58.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4389999999999999e-05 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366884+0000", 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013571781 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366969+0000", 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4728999999999995e-05 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367037+0000", 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7817999999999993e-05 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:886 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 180224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33531344000000002, 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013949134, 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 886 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7599999999999997e-07 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353321+0000", 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1239999999999999e-06 2024-09-11T09:28:58.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353327+0000", 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353332+0000", 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4499999999999997e-06 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367072+0000", 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013740148000000001 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367156+0000", 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4238000000000002e-05 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367268+0000", 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011187 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:887 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 475136~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33529466000000002, 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014108185000000001, 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 887 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353345+0000", 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4959999999999999e-06 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353352+0000", 2024-09-11T09:28:58.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6660000000000002e-06 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353357+0000", 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1880000000000001e-06 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367299+0000", 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013942090000000001 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367362+0000", 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2448999999999998e-05 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367446+0000", 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3886000000000003e-05 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:888 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353366+0000", 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33526627799999997, 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014376607, 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 888 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353366+0000", 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353366+0000", 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353367+0000", 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:28:58.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353369+0000", 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.057e-06 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353375+0000", 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8290000000000002e-06 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353393+0000", 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7167e-05 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367474+0000", 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014081529000000001 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367643+0000", 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000169059 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367743+0000", 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9372999999999999e-05 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:909 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 245760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353480+0000", 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.335152004, 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01445602, 2024-09-11T09:28:58.755 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 909 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353480+0000", 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353480+0000", 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353481+0000", 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353482+0000", 2024-09-11T09:28:58.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7260000000000001e-06 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353501+0000", 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8612999999999999e-05 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353507+0000", 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367772+0000", 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014264967 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367841+0000", 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8742e-05 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367936+0000", 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5254999999999995e-05 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.759 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:910 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 540672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353515+0000", 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33511696099999999, 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559327, 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 910 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353515+0000", 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353515+0000", 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353516+0000", 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353518+0000", 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353524+0000", 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4540000000000004e-06 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353529+0000", 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3850000000000004e-06 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367967+0000", 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014438493 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368019+0000", 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1958e-05 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368075+0000", 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5321000000000001e-05 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:911 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353534+0000", 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33509791799999999, 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014646932, 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 911 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353534+0000", 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353534+0000", 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353535+0000", 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1699999999999999e-07 2024-09-11T09:28:58.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353538+0000", 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0240000000000002e-06 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353543+0000", 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999997e-06 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353548+0000", 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0030000000000002e-06 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368102+0000", 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014553422 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368141+0000", 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9836000000000003e-05 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368181+0000", 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-05 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:919 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354026+0000", 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33460594500000002, 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014377974, 2024-09-11T09:28:58.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 919 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354026+0000", 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354026+0000", 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354027+0000", 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4799999999999997e-07 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354033+0000", 2024-09-11T09:28:58.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.173e-06 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354044+0000", 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0438000000000001e-05 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354051+0000", 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368225+0000", 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014174717 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368302+0000", 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6629000000000001e-05 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368404+0000", 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010226999999999999 2024-09-11T09:28:58.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:920 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33457040399999999, 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014518201, 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 920 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354065+0000", 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9129999999999999e-06 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354072+0000", 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5690000000000001e-06 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354084+0000", 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2386e-05 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368432+0000", 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014347258999999999 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368494+0000", 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2775999999999997e-05 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368580+0000", 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5749999999999997e-05 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:921 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33453365400000001, 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014651455000000001, 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 921 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:28:58.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354102+0000", 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354109+0000", 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354114+0000", 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4639999999999997e-06 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368599+0000", 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484566000000001 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368658+0000", 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9168e-05 2024-09-11T09:28:58.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368750+0000", 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2289999999999999e-05 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:922 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.33451170299999999, 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014771685, 2024-09-11T09:28:58.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 922 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:28:58.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354123+0000", 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.362e-06 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354129+0000", 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8690000000000001e-06 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354134+0000", 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.933e-06 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368786+0000", 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014651453 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368820+0000", 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4752999999999997e-05 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368892+0000", 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1901000000000002e-05 2024-09-11T09:28:58.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:28:58.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:28:58.788 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:58.790 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:28:58.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:28:58.791 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:58.789+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:683 punch from 0x53f2400 to 0x53f4c00, (0x2800 bytes) 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:684 trunc from 0xa086a00 to 0x86a2000 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:685 read 0x63ac400 thru 0x63b3dc2 (0x79c3 bytes) 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:687 read 0x6cad600 thru 0x6cba317 (0xcd18 bytes) 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:688 read 0x6b9f800 thru 0x6ba4ac0 (0x52c1 bytes) 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:689 trunc from 0x86a2000 to 0x3aee600 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:691 read 0x32e3400 thru 0x32edd1c (0xa91d bytes) 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:693 read 0x2c5a00 thru 0x2d36c8 (0xdcc9 bytes) 2024-09-11T09:28:58.792 INFO:teuthology.orchestra.run.smithi107.stdout:694 read 0x20e6a00 thru 0x20f57dc (0xeddd bytes) 2024-09-11T09:28:58.798 INFO:teuthology.orchestra.run.smithi107.stdout:696 write 0x76ce000 thru 0x76d9037 (0xb038 bytes) 2024-09-11T09:28:58.802 INFO:teuthology.orchestra.run.smithi107.stdout:701 trunc from 0x76d9200 to 0x4ea1400 2024-09-11T09:28:58.819 INFO:teuthology.orchestra.run.smithi107.stdout:703 read 0x1e9c00 thru 0x1ef0c1 (0x54c2 bytes) 2024-09-11T09:28:58.827 INFO:teuthology.orchestra.run.smithi107.stdout:707 read 0x2d87c00 thru 0x2d8aa0b (0x2e0c bytes) 2024-09-11T09:28:58.838 INFO:teuthology.orchestra.run.smithi107.stdout:710 trunc from 0x4ea1400 to 0xe43b200 2024-09-11T09:28:58.847 INFO:teuthology.orchestra.run.smithi107.stdout:713 punch from 0x76b1800 to 0x76be800, (0xd000 bytes) 2024-09-11T09:28:58.850 INFO:teuthology.orchestra.run.smithi107.stdout:716 write 0x1057a00 thru 0x10638a6 (0xbea7 bytes) 2024-09-11T09:28:58.863 INFO:teuthology.orchestra.run.smithi107.stdout:717 trunc from 0xe43b200 to 0x36bc800 2024-09-11T09:28:58.880 INFO:teuthology.orchestra.run.smithi107.stdout:719 read 0x1492e00 thru 0x149d304 (0xa505 bytes) 2024-09-11T09:28:58.889 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:58.890+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:58.911 INFO:teuthology.orchestra.run.smithi107.stdout:720 write 0xebf8200 thru 0xebf89cd (0x7ce bytes) 2024-09-11T09:28:58.918 INFO:teuthology.orchestra.run.smithi107.stdout:722 read 0xc1bda00 thru 0xc1ccc6a (0xf26b bytes) 2024-09-11T09:28:58.918 INFO:teuthology.orchestra.run.smithi107.stdout:723 read 0x7423e00 thru 0x742aa58 (0x6c59 bytes) 2024-09-11T09:28:58.919 INFO:teuthology.orchestra.run.smithi107.stdout:724 punch from 0x87d7800 to 0x87dbc00, (0x4400 bytes) 2024-09-11T09:28:58.919 INFO:teuthology.orchestra.run.smithi107.stdout:725 write 0x950e200 thru 0x950e4f6 (0x2f7 bytes) 2024-09-11T09:28:58.928 INFO:teuthology.orchestra.run.smithi107.stdout:726 trunc from 0xebf8a00 to 0x72e2800 2024-09-11T09:28:58.944 INFO:teuthology.orchestra.run.smithi107.stdout:727 trunc from 0x72e2800 to 0x5d0fc00 2024-09-11T09:28:58.947 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:58.947 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:58.948 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:28:58.948 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:58.958 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:28:58.960 INFO:teuthology.orchestra.run.smithi107.stdout:728 punch from 0x4f74c00 to 0x4f7f400, (0xa800 bytes) 2024-09-11T09:28:58.965 INFO:teuthology.orchestra.run.smithi107.stdout:729 write 0x1f3e00 thru 0x1fd70b (0x990c bytes) 2024-09-11T09:28:58.966 INFO:teuthology.orchestra.run.smithi107.stdout:730 read 0x705200 thru 0x70b237 (0x6038 bytes) 2024-09-11T09:28:58.966 INFO:teuthology.orchestra.run.smithi107.stdout:731 punch from 0x2d26e00 to 0x2d36c00, (0xfe00 bytes) 2024-09-11T09:28:58.967 INFO:teuthology.orchestra.run.smithi107.stdout:732 read 0x1d9da00 thru 0x1da40f6 (0x66f7 bytes) 2024-09-11T09:28:58.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:28:58.969 INFO:teuthology.orchestra.run.smithi107.stdout:733 punch from 0x28fe800 to 0x290e000, (0xf800 bytes) 2024-09-11T09:28:58.969 INFO:teuthology.orchestra.run.smithi107.stdout:734 read 0x4f28800 thru 0x4f2a5c0 (0x1dc1 bytes) 2024-09-11T09:28:58.979 INFO:teuthology.orchestra.run.smithi107.stdout:736 write 0x793c000 thru 0x794578e (0x978f bytes) 2024-09-11T09:28:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:737 trunc from 0x7945800 to 0x660800 2024-09-11T09:28:58.999 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:59.000+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:59.047 INFO:teuthology.orchestra.run.smithi107.stdout:738 write 0xeb33800 thru 0xeb36116 (0x2917 bytes) 2024-09-11T09:28:59.048 INFO:teuthology.orchestra.run.smithi107.stdout:742 read 0x9a5ac00 thru 0x9a6315b (0x855c bytes) 2024-09-11T09:28:59.061 INFO:teuthology.orchestra.run.smithi107.stdout:746 trunc from 0xeb36200 to 0x3162400 2024-09-11T09:28:59.089 INFO:teuthology.orchestra.run.smithi107.stdout:748 write 0x4e29000 thru 0x4e2e5d6 (0x55d7 bytes) 2024-09-11T09:28:59.091 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:59.092+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:59.091 INFO:teuthology.orchestra.run.smithi107.stdout:749 trunc from 0x4e2e600 to 0x6495200 2024-09-11T09:28:59.099 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:28:59.099 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:28:59.109 INFO:teuthology.orchestra.run.smithi107.stdout:756 read 0x5493c00 thru 0x549c184 (0x8585 bytes) 2024-09-11T09:28:59.109 INFO:teuthology.orchestra.run.smithi107.stdout:761 read 0x1548600 thru 0x1552d41 (0xa742 bytes) 2024-09-11T09:28:59.110 INFO:teuthology.orchestra.run.smithi107.stdout:762 punch from 0x2418600 to 0x2423400, (0xae00 bytes) 2024-09-11T09:28:59.111 INFO:teuthology.orchestra.run.smithi107.stdout:765 write 0x138f000 thru 0x1394145 (0x5146 bytes) 2024-09-11T09:28:59.111 INFO:teuthology.orchestra.run.smithi107.stdout:766 read 0x277ea00 thru 0x2780c8e (0x228f bytes) 2024-09-11T09:28:59.111 INFO:teuthology.orchestra.run.smithi107.stdout:768 punch from 0x4c3e000 to 0x4c46e00, (0x8e00 bytes) 2024-09-11T09:28:59.112 INFO:teuthology.orchestra.run.smithi107.stdout:769 punch from 0x5668000 to 0x566de00, (0x5e00 bytes) 2024-09-11T09:28:59.116 INFO:teuthology.orchestra.run.smithi107.stdout:770 trunc from 0x6495200 to 0x9c88400 2024-09-11T09:28:59.118 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:28:59.118 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:28:59.120 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:28:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:28:59.120 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:28:59.121 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:28:59.121 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:28:59.131 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:28:59.135 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:28:59.135 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:28:59.138 INFO:teuthology.orchestra.run.smithi107.stdout:775 write 0x820a00 thru 0x82c4eb (0xbaec bytes) 2024-09-11T09:28:59.140 INFO:teuthology.orchestra.run.smithi107.stdout:776 write 0x4942e00 thru 0x494b9bd (0x8bbe bytes) 2024-09-11T09:28:59.141 INFO:teuthology.orchestra.run.smithi107.stdout:777 write 0x716fc00 thru 0x716fd76 (0x177 bytes) 2024-09-11T09:28:59.141 INFO:teuthology.orchestra.run.smithi107.stdout:778 punch from 0x116800 to 0x120e00, (0xa600 bytes) 2024-09-11T09:28:59.151 INFO:teuthology.orchestra.run.smithi107.stdout:783 trunc from 0x9c88400 to 0x16e8800 2024-09-11T09:28:59.154 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:28:59.154 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:28:59.174 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:28:59.174 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:28:59.185 INFO:teuthology.orchestra.run.smithi107.stdout:784 write 0x1d9e600 thru 0x1da4fc4 (0x69c5 bytes) 2024-09-11T09:28:59.189 INFO:teuthology.orchestra.run.smithi107.stdout:790 write 0x18b7e00 thru 0x18ba3f5 (0x25f6 bytes) 2024-09-11T09:28:59.190 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:28:59.191 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:28:59.191 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:59.192+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:59.215 INFO:teuthology.orchestra.run.smithi107.stdout:794 write 0x9dcd400 thru 0x9dcee7f (0x1a80 bytes) 2024-09-11T09:28:59.233 INFO:teuthology.orchestra.run.smithi107.stdout:795 write 0xe301600 thru 0xe310882 (0xf283 bytes) 2024-09-11T09:28:59.233 INFO:teuthology.orchestra.run.smithi107.stdout:796 read 0x4591400 thru 0x45985b3 (0x71b4 bytes) 2024-09-11T09:28:59.233 INFO:teuthology.orchestra.run.smithi107.stdout:799 write 0x234e600 thru 0x2357ba2 (0x95a3 bytes) 2024-09-11T09:28:59.235 INFO:teuthology.orchestra.run.smithi107.stdout:800 read 0x6fb400 thru 0x6fbdf9 (0x9fa bytes) 2024-09-11T09:28:59.235 INFO:teuthology.orchestra.run.smithi107.stdout:802 write 0xcbe6a00 thru 0xcbf2d45 (0xc346 bytes) 2024-09-11T09:28:59.249 INFO:teuthology.orchestra.run.smithi107.stdout:805 trunc from 0xe310a00 to 0x2726800 2024-09-11T09:28:59.253 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:59.253 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:59.264 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:28:59.274 INFO:teuthology.orchestra.run.smithi107.stdout:806 punch from 0x19c2800 to 0x19c3600, (0xe00 bytes) 2024-09-11T09:28:59.276 INFO:teuthology.orchestra.run.smithi107.stdout:812 punch from 0x225fa00 to 0x226d200, (0xd800 bytes) 2024-09-11T09:28:59.276 INFO:teuthology.orchestra.run.smithi107.stdout:815 trunc from 0x2726800 to 0x2c0e800 2024-09-11T09:28:59.288 INFO:teuthology.orchestra.run.smithi107.stdout:817 punch from 0x100a00 to 0x109600, (0x8c00 bytes) 2024-09-11T09:28:59.290 INFO:teuthology.orchestra.run.smithi107.stdout:818 clone 5 order 19 su 65536 sc 15 2024-09-11T09:28:59.305 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:28:59.306+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:28:59.374 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:59.374 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:59.382 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:28:59.393 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:59.393+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:59.489 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:59.541 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:59.543 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:28:59.543 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:59.543+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:59.643 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:28:59.644+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:28:59.690 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:59.690 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:59.698 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:28:59.744 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:59.744+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:28:59.794 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone4 from 0x2c0e800 (overlap 0x62d800) to 0x453000 2024-09-11T09:28:59.809 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:59.810 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:59.819 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:28:59.863 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:28:59.864+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:28:59.928 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:28:59.928 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:28:59.936 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:28:59.946 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:28:59.947+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:00.047 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:00.047+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:00.085 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:00.085 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:00.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:00.099 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-11T09:29:00.148 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:00.148+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:00.223 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:00.223 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:00.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:00.249 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:00.249+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:00.350 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:00.350+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:00.413 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:00.414 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:00.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:00.451 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:00.451+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:00.552 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:00.553+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:00.574 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:00.574 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:00.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:00.653 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:00.654+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:00.680 INFO:teuthology.orchestra.run.smithi107.stdout:819 read 0x14dc00 thru 0x15c116 (0xe517 bytes) 2024-09-11T09:29:00.682 INFO:teuthology.orchestra.run.smithi107.stdout:820 clone 6 order 23 su 65536 sc 2 2024-09-11T09:29:00.687 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:00.687 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:00.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:00.754 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:00.754+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:00.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:29:00.811 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone5 from 0x2c0e800 (overlap 0x2c0e800) to 0x467c00 2024-09-11T09:29:00.855 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:00.855+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:00.872 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:00.872 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:00.886 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:00.955 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:00.956+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:01.019 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:01.019 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:01.029 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:01.056 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:01.057+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:01.157 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:01.158+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:01.179 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:01.179 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:01.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:01.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:01.271 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:01.272+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:01.277 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-11T09:29:01.330 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:01.331 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:01.339 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:01.340 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:01.348 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:01.349 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:01.349 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:01.359 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:01.359+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:01.365 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:01.365 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:01.381 INFO:teuthology.orchestra.run.smithi107.stdout:821 write 0xc961800 thru 0xc967968 (0x6169 bytes) 2024-09-11T09:29:01.382 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:01.382 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:01.382 INFO:teuthology.orchestra.run.smithi107.stdout:823 write 0xb134800 thru 0xb1363dc (0x1bdd bytes) 2024-09-11T09:29:01.383 INFO:teuthology.orchestra.run.smithi107.stdout:824 read 0xbe29400 thru 0xbe2c648 (0x3249 bytes) 2024-09-11T09:29:01.397 INFO:teuthology.orchestra.run.smithi107.stdout:826 write 0xe640a00 thru 0xe64e7a3 (0xdda4 bytes) 2024-09-11T09:29:01.397 INFO:teuthology.orchestra.run.smithi107.stdout:828 read 0x1903600 thru 0x190779e (0x419f bytes) 2024-09-11T09:29:01.398 INFO:teuthology.orchestra.run.smithi107.stdout:829 punch from 0x9a0b000 to 0x9a0ee00, (0x3e00 bytes) 2024-09-11T09:29:01.398 INFO:teuthology.orchestra.run.smithi107.stdout:831 write 0xddcb400 thru 0xddcf917 (0x4518 bytes) 2024-09-11T09:29:01.398 INFO:teuthology.orchestra.run.smithi107.stdout:833 write 0xabe9e00 thru 0xabebd06 (0x1f07 bytes) 2024-09-11T09:29:01.399 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:01.399 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:01.411 INFO:teuthology.orchestra.run.smithi107.stdout:835 write 0xe8f3200 thru 0xe8fe946 (0xb747 bytes) 2024-09-11T09:29:01.418 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:01.418 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:01.423 INFO:teuthology.orchestra.run.smithi107.stdout:836 trunc from 0xe8fea00 to 0x5151000 2024-09-11T09:29:01.439 INFO:teuthology.orchestra.run.smithi107.stdout:837 read 0x334ba00 thru 0x334fd0e (0x430f bytes) 2024-09-11T09:29:01.450 INFO:teuthology.orchestra.run.smithi107.stdout:838 write 0x6f54800 thru 0x6f57558 (0x2d59 bytes) 2024-09-11T09:29:01.450 INFO:teuthology.orchestra.run.smithi107.stdout:839 read 0x2518e00 thru 0x251e405 (0x5606 bytes) 2024-09-11T09:29:01.451 INFO:teuthology.orchestra.run.smithi107.stdout:840 write 0x3225000 thru 0x322a2bb (0x52bc bytes) 2024-09-11T09:29:01.451 INFO:teuthology.orchestra.run.smithi107.stdout:842 read 0x1eea800 thru 0x1eeb0f2 (0x8f3 bytes) 2024-09-11T09:29:01.455 INFO:teuthology.orchestra.run.smithi107.stdout:844 punch from 0x2566400 to 0x2571e00, (0xba00 bytes) 2024-09-11T09:29:01.459 INFO:teuthology.orchestra.run.smithi107.stdout:845 read 0x6135600 thru 0x6137262 (0x1c63 bytes) 2024-09-11T09:29:01.460 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:01.460+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:01.461 INFO:teuthology.orchestra.run.smithi107.stdout:847 write 0x28bd800 thru 0x28c3e86 (0x6687 bytes) 2024-09-11T09:29:01.467 INFO:teuthology.orchestra.run.smithi107.stdout:849 trunc from 0x6f57600 to 0x1c03800 2024-09-11T09:29:01.529 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:01.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:835 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 180224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1758935240000001, 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012798782, 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 835 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352431+0000", 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5699999999999998e-07 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352433+0000", 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9740000000000001e-06 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352439+0000", 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7660000000000004e-06 2024-09-11T09:29:01.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352443+0000", 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1910000000000002e-06 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365065+0000", 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012622517 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365134+0000", 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8604999999999998e-05 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365229+0000", 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5371999999999994e-05 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:836 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 475136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:29:01.534 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.175875665, 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012941388, 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 836 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352449+0000", 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352451+0000", 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.832e-06 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352456+0000", 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3380000000000004e-06 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352461+0000", 2024-09-11T09:29:01.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6319999999999999e-06 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365259+0000", 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012797894000000001 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365327+0000", 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8069000000000002e-05 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365390+0000", 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3246999999999999e-05 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:837 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352466+0000", 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1758577790000002, 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013107196999999999, 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 837 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352466+0000", 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352466+0000", 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352467+0000", 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.84e-07 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352469+0000", 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9860000000000001e-06 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352474+0000", 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3519999999999997e-06 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352479+0000", 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.459e-06 2024-09-11T09:29:01.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365420+0000", 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.012941104 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365486+0000", 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6082999999999997e-05 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365574+0000", 2024-09-11T09:29:01.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7829000000000002e-05 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:849 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 147456~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352484+0000", 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1758398849999998, 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013236056, 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 849 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352484+0000", 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352484+0000", 2024-09-11T09:29:01.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352485+0000", 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6399999999999998e-07 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352490+0000", 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6670000000000001e-06 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352496+0000", 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9869999999999996e-06 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352501+0000", 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3800000000000004e-06 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365604+0000", 2024-09-11T09:29:01.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013103571 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365666+0000", 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1983999999999995e-05 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365720+0000", 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4103e-05 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:850 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 442368~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.175799638, 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013366031, 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 850 2024-09-11T09:29:01.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352525+0000", 2024-09-11T09:29:01.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5000000000000003e-07 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352527+0000", 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1710000000000001e-06 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352534+0000", 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5050000000000004e-06 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352538+0000", 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6550000000000003e-06 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365748+0000", 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013209594 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365797+0000", 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9409000000000001e-05 2024-09-11T09:29:01.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365891+0000", 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3146999999999994e-05 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:851 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 737280~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352545+0000", 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1757791869999998, 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013868831, 2024-09-11T09:29:01.553 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 851 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352545+0000", 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352545+0000", 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352546+0000", 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352547+0000", 2024-09-11T09:29:01.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9130000000000001e-06 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352553+0000", 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9229999999999999e-06 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352558+0000", 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365919+0000", 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013361708999999999 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.365985+0000", 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5562999999999996e-05 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366414+0000", 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042899000000000002 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:852 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1757546219999999, 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013986192999999999, 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 852 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.558 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352570+0000", 2024-09-11T09:29:01.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352744+0000", 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017385299999999999 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352754+0000", 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0334e-05 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.352761+0000", 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6279999999999999e-06 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366442+0000", 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013681415000000001 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366493+0000", 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0263000000000003e-05 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366556+0000", 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3176000000000006e-05 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:872 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 212992~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1750712999999999, 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013469549000000001, 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.562 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 872 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353253+0000", 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353255+0000", 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9929999999999998e-06 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353261+0000", 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0639999999999999e-06 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353266+0000", 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5410000000000002e-06 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366589+0000", 2024-09-11T09:29:01.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013323536 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366651+0000", 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1544999999999994e-05 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366722+0000", 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1459999999999997e-05 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.566 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:873 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 507904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1750525949999999, 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013566754, 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 873 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.567 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353272+0000", 2024-09-11T09:29:01.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353274+0000", 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9209999999999999e-06 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353280+0000", 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0329999999999997e-06 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353285+0000", 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5940000000000004e-06 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366748+0000", 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01346346 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366786+0000", 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7812999999999999e-05 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366838+0000", 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2571999999999997e-05 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:874 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1750343170000002, 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01374672, 2024-09-11T09:29:01.571 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 874 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353290+0000", 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2399999999999999e-07 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353292+0000", 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.807e-06 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353298+0000", 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353312+0000", 2024-09-11T09:29:01.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4389999999999999e-05 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366884+0000", 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013571781 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.366969+0000", 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4728999999999995e-05 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367037+0000", 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7817999999999993e-05 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:886 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 180224~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1750054040000002, 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013949134, 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 886 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353319+0000", 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7599999999999997e-07 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353321+0000", 2024-09-11T09:29:01.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1239999999999999e-06 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353327+0000", 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8279999999999999e-06 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353332+0000", 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4499999999999997e-06 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367072+0000", 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013740148000000001 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367156+0000", 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4238000000000002e-05 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367268+0000", 2024-09-11T09:29:01.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011187 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:887 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 475136~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1749866240000002, 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014108185000000001, 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.581 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 887 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353338+0000", 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353345+0000", 2024-09-11T09:29:01.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4959999999999999e-06 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353352+0000", 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6660000000000002e-06 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353357+0000", 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1880000000000001e-06 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367299+0000", 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.013942090000000001 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367362+0000", 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2448999999999998e-05 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367446+0000", 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3886000000000003e-05 2024-09-11T09:29:01.585 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:888 3.3 3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:6 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353366+0000", 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1749582420000002, 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014376607, 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.586 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 888 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353366+0000", 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353366+0000", 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353367+0000", 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353369+0000", 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.057e-06 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353375+0000", 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8290000000000002e-06 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353393+0000", 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7167e-05 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367474+0000", 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014081529000000001 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367643+0000", 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000169059 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367743+0000", 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9372999999999999e-05 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:909 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 245760~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.590 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353480+0000", 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1748439679999998, 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.01445602, 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 909 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353480+0000", 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353480+0000", 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353481+0000", 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3799999999999998e-07 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353482+0000", 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7260000000000001e-06 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353501+0000", 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8612999999999999e-05 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353507+0000", 2024-09-11T09:29:01.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3790000000000003e-06 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367772+0000", 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014264967 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367841+0000", 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8742e-05 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367936+0000", 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5254999999999995e-05 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:910 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 540672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353515+0000", 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1748089249999998, 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014559327, 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 910 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353515+0000", 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353515+0000", 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353516+0000", 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7700000000000004e-07 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353518+0000", 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1390000000000002e-06 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353524+0000", 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4540000000000004e-06 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353529+0000", 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3850000000000004e-06 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.367967+0000", 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014438493 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368019+0000", 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1958e-05 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368075+0000", 2024-09-11T09:29:01.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5321000000000001e-05 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:911 3.3 3:c725f596:::rbd_data.11062c268ffd.0000000000000007:6 [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.353534+0000", 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1747898819999998, 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014646932, 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.600 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 911 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353534+0000", 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353534+0000", 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353535+0000", 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1699999999999999e-07 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353538+0000", 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0240000000000002e-06 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353543+0000", 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999997e-06 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.353548+0000", 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0030000000000002e-06 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368102+0000", 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014553422 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368141+0000", 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9836000000000003e-05 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368181+0000", 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-05 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:919 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354026+0000", 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1742979089999999, 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014377974, 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 919 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354026+0000", 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354026+0000", 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354027+0000", 2024-09-11T09:29:01.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4799999999999997e-07 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354033+0000", 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.173e-06 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354044+0000", 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0438000000000001e-05 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354051+0000", 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368225+0000", 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014174717 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368302+0000", 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6629000000000001e-05 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368404+0000", 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010226999999999999 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:920 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1742623679999999, 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014518201, 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 920 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354062+0000", 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4799999999999998e-07 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354065+0000", 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9129999999999999e-06 2024-09-11T09:29:01.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354072+0000", 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5690000000000001e-06 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354084+0000", 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2386e-05 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368432+0000", 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014347258999999999 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368494+0000", 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2775999999999997e-05 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368580+0000", 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5749999999999997e-05 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:921 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.1742256179999999, 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014651455000000001, 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 921 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354099+0000", 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9500000000000002e-07 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354102+0000", 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8339999999999999e-06 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354109+0000", 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354114+0000", 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4639999999999997e-06 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368599+0000", 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014484566000000001 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368658+0000", 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9168e-05 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368750+0000", 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2289999999999999e-05 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4546.0:922 3.3 3:c0ea1cec:::rbd_header.1106b949ffb0:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.174203667, 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014771685, 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4546", 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/564726233", 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 922 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354121+0000", 2024-09-11T09:29:01.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354123+0000", 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.362e-06 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354129+0000", 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8690000000000001e-06 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.354134+0000", 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.933e-06 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368786+0000", 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.014651453 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368820+0000", 2024-09-11T09:29:01.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4752999999999997e-05 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:58.368892+0000", 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1901000000000002e-05 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.623 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:01.625 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:01.625+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:850 write 0x5179400 thru 0x5188660 (0xf261 bytes) 2024-09-11T09:29:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:854 write 0xe675400 thru 0xe68116c (0xbd6d bytes) 2024-09-11T09:29:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:29:01.625 INFO:teuthology.orchestra.run.smithi107.stdout:858 punch from 0x29cc00 to 0x2a1400, (0x4800 bytes) 2024-09-11T09:29:01.626 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:01.636 INFO:teuthology.orchestra.run.smithi107.stdout:859 trunc from 0xe681200 to 0x6cec00 2024-09-11T09:29:01.713 INFO:teuthology.orchestra.run.smithi107.stdout:861 trunc from 0x6cec00 to 0x2aaea00 2024-09-11T09:29:01.722 INFO:teuthology.orchestra.run.smithi107.stdout:864 write 0x2a94000 thru 0x2a98e08 (0x4e09 bytes) 2024-09-11T09:29:01.723 INFO:teuthology.orchestra.run.smithi107.stdout:865 read 0x264e600 thru 0x2654627 (0x6028 bytes) 2024-09-11T09:29:01.723 INFO:teuthology.orchestra.run.smithi107.stdout:867 read 0x22e4200 thru 0x22ea970 (0x6771 bytes) 2024-09-11T09:29:01.725 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:01.726+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:01.741 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:29:01.759 INFO:teuthology.orchestra.run.smithi107.stdout:868 write 0xd0bc400 thru 0xd0c7f00 (0xbb01 bytes) 2024-09-11T09:29:01.759 INFO:teuthology.orchestra.run.smithi107.stdout:869 write 0xb4f7600 thru 0xb500bde (0x95df bytes) 2024-09-11T09:29:01.759 INFO:teuthology.orchestra.run.smithi107.stdout:871 read 0x2e21e00 thru 0x2e2be79 (0xa07a bytes) 2024-09-11T09:29:01.759 INFO:teuthology.orchestra.run.smithi107.stdout:872 write 0xb8c7800 thru 0xb8d2642 (0xae43 bytes) 2024-09-11T09:29:01.760 INFO:teuthology.orchestra.run.smithi107.stdout:873 read 0x2445a00 thru 0x24505bc (0xabbd bytes) 2024-09-11T09:29:01.760 INFO:teuthology.orchestra.run.smithi107.stdout:874 read 0x3df3e00 thru 0x3dfbe73 (0x8074 bytes) 2024-09-11T09:29:01.760 INFO:teuthology.orchestra.run.smithi107.stdout:875 read 0xa88e200 thru 0xa8983cd (0xa1ce bytes) 2024-09-11T09:29:01.760 INFO:teuthology.orchestra.run.smithi107.stdout:876 read 0x883ac00 thru 0x884713c (0xc53d bytes) 2024-09-11T09:29:01.770 INFO:teuthology.orchestra.run.smithi107.stdout:878 trunc from 0xd0c8000 to 0x3450600 2024-09-11T09:29:01.783 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2762 3.e 3:7612e368:::rbd_header.1106259a042c:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-11T09:29:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.783833+0000", 2024-09-11T09:29:01.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00038531200000000001, 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00041053999999999998, 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2762 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:01.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783833+0000", 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783833+0000", 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783834+0000", 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4e-07 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783839+0000", 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2429999999999996e-06 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783851+0000", 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.17e-05 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783860+0000", 2024-09-11T09:29:01.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9390000000000003e-06 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783905+0000", 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5880000000000001e-05 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.783950+0000", 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4446000000000001e-05 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.784072+0000", 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000121698 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-11T09:29:01.788 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:01.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:01.794 INFO:teuthology.orchestra.run.smithi107.stdout:879 read 0x1603e00 thru 0x160b2e6 (0x74e7 bytes) 2024-09-11T09:29:01.811 INFO:teuthology.orchestra.run.smithi107.stdout:880 write 0x40fb400 thru 0x4101155 (0x5d56 bytes) 2024-09-11T09:29:01.812 INFO:teuthology.orchestra.run.smithi107.stdout:882 punch from 0x3138e00 to 0x3145c00, (0xce00 bytes) 2024-09-11T09:29:01.826 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:01.826+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:01.834 INFO:teuthology.orchestra.run.smithi107.stdout:887 write 0xad97a00 thru 0xad9f885 (0x7e86 bytes) 2024-09-11T09:29:01.834 INFO:teuthology.orchestra.run.smithi107.stdout:889 read 0x4fc7800 thru 0x4fd526b (0xda6c bytes) 2024-09-11T09:29:01.838 INFO:teuthology.orchestra.run.smithi107.stdout:892 trunc from 0xad9fa00 to 0x814b400 2024-09-11T09:29:01.862 INFO:teuthology.orchestra.run.smithi107.stdout:893 read 0x17dea00 thru 0x17e4614 (0x5c15 bytes) 2024-09-11T09:29:01.863 INFO:teuthology.orchestra.run.smithi107.stdout:894 write 0x69cf400 thru 0x69d8c70 (0x9871 bytes) 2024-09-11T09:29:01.863 INFO:teuthology.orchestra.run.smithi107.stdout:896 write 0x64bac00 thru 0x64c9153 (0xe554 bytes) 2024-09-11T09:29:01.863 INFO:teuthology.orchestra.run.smithi107.stdout:898 write 0x5208600 thru 0x5215535 (0xcf36 bytes) 2024-09-11T09:29:01.863 INFO:teuthology.orchestra.run.smithi107.stdout:899 write 0x282ea00 thru 0x283cd79 (0xe37a bytes) 2024-09-11T09:29:01.865 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:01.865 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:01.866 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-11T09:29:01.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-11T09:29:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:900 trunc from 0x814b400 to 0x6f9000 2024-09-11T09:29:01.904 INFO:teuthology.orchestra.run.smithi107.stdout:902 punch from 0x465800 to 0x474000, (0xe800 bytes) 2024-09-11T09:29:01.909 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:01.909 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:01.909 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:01.909 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:01.909 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:01.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:01.924 INFO:teuthology.orchestra.run.smithi107.stdout:903 write 0x4f0d600 thru 0x4f1bb01 (0xe502 bytes) 2024-09-11T09:29:01.924 INFO:teuthology.orchestra.run.smithi107.stdout:904 write 0x4205a00 thru 0x4208943 (0x2f44 bytes) 2024-09-11T09:29:01.924 INFO:teuthology.orchestra.run.smithi107.stdout:905 read 0x2c8600 thru 0x2c8afe (0x4ff bytes) 2024-09-11T09:29:01.925 INFO:teuthology.orchestra.run.smithi107.stdout:908 read 0x353d400 thru 0x3549f75 (0xcb76 bytes) 2024-09-11T09:29:01.925 INFO:teuthology.orchestra.run.smithi107.stdout:911 read 0x28a0a00 thru 0x28a1d62 (0x1363 bytes) 2024-09-11T09:29:01.925 INFO:teuthology.orchestra.run.smithi107.stdout:914 read 0x2f3d800 thru 0x2f42893 (0x5094 bytes) 2024-09-11T09:29:01.926 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:01.927+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:01.948 INFO:teuthology.orchestra.run.smithi107.stdout:915 write 0x8177c00 thru 0x81819e1 (0x9de2 bytes) 2024-09-11T09:29:01.949 INFO:teuthology.orchestra.run.smithi107.stdout:917 punch from 0x7e9ce00 to 0x7ea6800, (0x9a00 bytes) 2024-09-11T09:29:01.949 INFO:teuthology.orchestra.run.smithi107.stdout:918 read 0x43ca600 thru 0x43d7ccc (0xd6cd bytes) 2024-09-11T09:29:01.950 INFO:teuthology.orchestra.run.smithi107.stdout:919 read 0x54b0a00 thru 0x54b8444 (0x7a45 bytes) 2024-09-11T09:29:01.950 INFO:teuthology.orchestra.run.smithi107.stdout:921 read 0x3221c00 thru 0x3224ea5 (0x32a6 bytes) 2024-09-11T09:29:01.954 INFO:teuthology.orchestra.run.smithi107.stdout:922 trunc from 0x8181a00 to 0xb605400 2024-09-11T09:29:01.971 INFO:teuthology.orchestra.run.smithi107.stdout:923 punch from 0x52e5800 to 0x52ef600, (0x9e00 bytes) 2024-09-11T09:29:01.972 INFO:teuthology.orchestra.run.smithi107.stdout:924 clone 7 order 19 su 2048 sc 13 2024-09-11T09:29:02.006 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:02.007 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:02.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:02.010 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:02.010 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-11T09:29:02.031 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:02.031 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2479 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632834+0000", 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39840722899999997, 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028843035, 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2479 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632834+0000", 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632834+0000", 2024-09-11T09:29:02.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632835+0000", 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1999999999999999e-07 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632839+0000", 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632852+0000", 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2809000000000001e-05 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632859+0000", 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3499999999999999e-06 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661497+0000", 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028637493999999999 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661586+0000", 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9289999999999994e-05 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661677+0000", 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1377000000000003e-05 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2483 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1064960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39833877099999998, 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029001923999999998, 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2483 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632910+0000", 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1069999999999999e-06 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632917+0000", 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9670000000000004e-06 2024-09-11T09:29:02.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632922+0000", 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0429999999999998e-06 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661722+0000", 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028799495000000001 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661813+0000", 2024-09-11T09:29:02.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1118000000000004e-05 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661905+0000", 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1679000000000001e-05 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2492 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39829110299999998, 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029187141, 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2492 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:02.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.96e-07 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632953+0000", 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0980000000000001e-06 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632959+0000", 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.212e-06 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632964+0000", 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661950+0000", 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028986369000000001 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662047+0000", 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6718999999999993e-05 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662138+0000", 2024-09-11T09:29:02.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0488999999999994e-05 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2493 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1720320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632970+0000", 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39827121300000001, 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029393354999999999, 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2493 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632970+0000", 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632970+0000", 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632971+0000", 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632973+0000", 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632979+0000", 2024-09-11T09:29:02.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000004e-06 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632985+0000", 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662182+0000", 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029197330000000001 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662272+0000", 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9272999999999997e-05 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662364+0000", 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1998999999999998e-05 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2496 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39825001399999999, 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029611141000000001, 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2496 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5600000000000001e-07 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632994+0000", 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633000+0000", 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.643e-06 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633004+0000", 2024-09-11T09:29:02.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5820000000000003e-06 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662409+0000", 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029404354000000001 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662510+0000", 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101547 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662603+0000", 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2357999999999999e-05 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2497 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 966656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39823089, 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029818801999999998, 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2497 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633013+0000", 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0990000000000001e-06 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633019+0000", 2024-09-11T09:29:02.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0170000000000002e-06 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633024+0000", 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6410000000000001e-06 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662645+0000", 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029621056 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662743+0000", 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8292e-05 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662830+0000", 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6324999999999998e-05 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2500 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 966656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39814582599999998, 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029839158000000001, 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2500 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.06e-07 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633099+0000", 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9069999999999999e-06 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633107+0000", 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7370000000000006e-06 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633113+0000", 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7400000000000001e-06 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662847+0000", 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029734159999999999 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662896+0000", 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-05 2024-09-11T09:29:02.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662935+0000", 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9258000000000003e-05 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2506 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1409024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633141+0000", 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39810043299999998, 2024-09-11T09:29:02.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030041090999999999, 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2506 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633141+0000", 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633141+0000", 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633142+0000", 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.32e-07 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633144+0000", 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633150+0000", 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633155+0000", 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7740000000000004e-06 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662976+0000", 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029821070000000002 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663082+0000", 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105512 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663182+0000", 2024-09-11T09:29:02.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010068699999999999 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2511 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1622016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633160+0000", 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39808151400000003, 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030109462, 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2511 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633160+0000", 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633160+0000", 2024-09-11T09:29:02.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633161+0000", 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2100000000000002e-07 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633167+0000", 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2260000000000002e-06 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633173+0000", 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2230000000000004e-06 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633178+0000", 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7589999999999997e-06 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663194+0000", 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030016602999999999 2024-09-11T09:29:02.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663233+0000", 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9014999999999998e-05 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663270+0000", 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6214999999999998e-05 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2514 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39805771699999998, 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030311439999999999, 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2514 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633186+0000", 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633192+0000", 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0990000000000004e-06 2024-09-11T09:29:02.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633197+0000", 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663309+0000", 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03011229 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663407+0000", 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7990000000000002e-05 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663495+0000", 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8140999999999995e-05 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2516 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1622016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39803478799999997, 2024-09-11T09:29:02.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030377168, 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2516 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633210+0000", 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5600000000000001e-06 2024-09-11T09:29:02.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633217+0000", 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633222+0000", 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369999999999998e-06 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663508+0000", 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030285362 2024-09-11T09:29:02.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663546+0000", 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8631999999999999e-05 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663584+0000", 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7688000000000002e-05 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.082 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2523 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39793384900000001, 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030473742000000002, 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2523 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:02.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8299999999999998e-07 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633310+0000", 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633316+0000", 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633321+0000", 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663629+0000", 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030308348999999998 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663709+0000", 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0062999999999996e-05 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663782+0000", 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2169000000000006e-05 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2528 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1736704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633327+0000", 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39791433599999998, 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030671588999999999, 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2528 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633327+0000", 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633327+0000", 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633328+0000", 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633330+0000", 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.165e-06 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633336+0000", 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0140000000000004e-06 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633340+0000", 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000001e-06 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663826+0000", 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030485760000000001 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663917+0000", 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0370000000000004e-05 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663999+0000", 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2287999999999995e-05 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2530 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1851392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39789468, 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030819236999999999, 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2530 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.39e-07 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633349+0000", 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9630000000000001e-06 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633355+0000", 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5230000000000003e-06 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633359+0000", 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4889999999999997e-06 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664021+0000", 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030661702999999998 2024-09-11T09:29:02.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664097+0000", 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6320000000000001e-05 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664166+0000", 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8800000000000005e-05 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2533 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1753088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633385+0000", 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39785625000000002, 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030977601, 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2533 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633385+0000", 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633385+0000", 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633386+0000", 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999999e-07 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633388+0000", 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.12e-06 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633394+0000", 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633399+0000", 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7600000000000002e-06 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664204+0000", 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030805018 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664286+0000", 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2503000000000006e-05 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664363+0000", 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6760000000000004e-05 2024-09-11T09:29:02.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2536 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1753088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633456+0000", 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39778560299999999, 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031090034999999999, 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2536 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633456+0000", 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633456+0000", 2024-09-11T09:29:02.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633457+0000", 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7100000000000001e-07 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633460+0000", 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8700000000000001e-06 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633467+0000", 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7789999999999996e-06 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633473+0000", 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1340000000000001e-06 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664398+0000", 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030924423999999999 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664475+0000", 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7418999999999999e-05 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664546+0000", 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0737999999999999e-05 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2540 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39776107300000002, 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031258706999999997, 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2540 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633483+0000", 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1849999999999998e-06 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633490+0000", 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0419999999999997e-06 2024-09-11T09:29:02.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633496+0000", 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5709999999999997e-06 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664583+0000", 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031087571000000001 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664663+0000", 2024-09-11T09:29:02.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9374000000000003e-05 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664739+0000", 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6538999999999993e-05 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2543 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1785856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633503+0000", 2024-09-11T09:29:02.111 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39773814400000002, 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031419903999999999, 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2543 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633503+0000", 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633503+0000", 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633504+0000", 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633511+0000", 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7510000000000002e-06 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633518+0000", 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9639999999999997e-06 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633523+0000", 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664774+0000", 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031250759000000003 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664850+0000", 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5644999999999998e-05 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664923+0000", 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3747999999999999e-05 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2544 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.39770977299999999, 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031595493000000002, 2024-09-11T09:29:02.116 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2544 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633534+0000", 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1380000000000001e-06 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633541+0000", 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0550000000000002e-06 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633547+0000", 2024-09-11T09:29:02.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2970000000000003e-06 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664961+0000", 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031414624000000002 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665047+0000", 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5527999999999998e-05 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665127+0000", 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0390999999999997e-05 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2548 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633553+0000", 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.397688445, 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031761585000000002, 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:02.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2548 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633553+0000", 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633553+0000", 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633554+0000", 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1199999999999998e-07 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:02.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633556+0000", 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.024e-06 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633562+0000", 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4629999999999998e-06 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633567+0000", 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1000000000000003e-06 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665162+0000", 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031594672999999997 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665243+0000", 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1167000000000004e-05 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665315+0000", 2024-09-11T09:29:02.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1746000000000001e-05 2024-09-11T09:29:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:02.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:02.126 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:02.127 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:02.051+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:02.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:02.129 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:02.129+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:02.229 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:02.230+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:02.235 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:02.235 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:02.235 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:02.235 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:02.248 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:02.267 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:02.268 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:02.270 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:02.270 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-11T09:29:02.330 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:02.331+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:02.356 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:02.356 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:02.356 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:02.356 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:02.356 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:02.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:02.431 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:02.432+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:02.448 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:02.448 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:02.448 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:02.448 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:02.449 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:02.451 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:02.451 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:02.452 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-11T09:29:02.476 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:02.476 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:02.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:02.476 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:02.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1477 3.1 3:8330dad4:::rbd_data.1106b949ffb0.000000000000000c:7 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508524+0000", 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.96711781100000005, 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013641992, 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1477 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.477 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508524+0000", 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508524+0000", 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508524+0000", 2024-09-11T09:29:02.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6300000000000005e-07 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508527+0000", 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0070000000000002e-06 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508534+0000", 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1119999999999998e-06 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508540+0000", 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6869999999999999e-06 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.521971+0000", 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013430881 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522058+0000", 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7492999999999997e-05 2024-09-11T09:29:02.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522166+0000", 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107249 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1502 3.1 3:8330dad4:::rbd_data.1106b949ffb0.000000000000000c:7 [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508553+0000", 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.96708872000000001, 2024-09-11T09:29:02.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013758873, 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1502 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508553+0000", 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508553+0000", 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508553+0000", 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1600000000000001e-07 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508556+0000", 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5239999999999999e-06 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508567+0000", 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0934999999999999e-05 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508574+0000", 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1550000000000001e-06 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522201+0000", 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013627267 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522256+0000", 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5145000000000001e-05 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522312+0000", 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5231e-05 2024-09-11T09:29:02.485 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1503 3.1 3:8330dad4:::rbd_data.1106b949ffb0.000000000000000c:7 [read 1474560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508582+0000", 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.96705922899999996, 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013970731, 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.486 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1503 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508582+0000", 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508582+0000", 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508583+0000", 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4799999999999999e-07 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508591+0000", 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4319999999999999e-06 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508599+0000", 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3710000000000002e-06 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508609+0000", 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.064e-05 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522349+0000", 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013740304 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522440+0000", 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0242999999999996e-05 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522553+0000", 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113293 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1511 3.1 3:8330dad4:::rbd_data.1106b949ffb0.000000000000000c:7 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508624+0000", 2024-09-11T09:29:02.490 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.96701728200000003, 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014065284000000001, 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1511 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508624+0000", 2024-09-11T09:29:02.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508624+0000", 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508625+0000", 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0699999999999997e-07 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508627+0000", 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5940000000000001e-06 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508634+0000", 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8759999999999999e-06 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508642+0000", 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4159999999999998e-06 2024-09-11T09:29:02.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522590+0000", 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013948409 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522634+0000", 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4121e-05 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522689+0000", 2024-09-11T09:29:02.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5361000000000001e-05 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1514 3.1 3:8330dad4:::rbd_data.1106b949ffb0.000000000000000c:7 [read 1605632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508649+0000", 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.96699228400000004, 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014267702, 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.495 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1514 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508649+0000", 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.496 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508649+0000", 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508650+0000", 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8500000000000002e-07 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508652+0000", 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0779999999999998e-06 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508658+0000", 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2020000000000001e-06 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508663+0000", 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2490000000000001e-06 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522715+0000", 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014051398 2024-09-11T09:29:02.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522805+0000", 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0078000000000001e-05 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522917+0000", 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000112212 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1520 3.1f 3:fe6b904e:::rbd_data.1106259a042c.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.540177+0000", 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.93546428299999995, 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017952165999999999, 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1520 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.500 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.540177+0000", 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.540177+0000", 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.501 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.540178+0000", 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9999999999999996e-07 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.547922+0000", 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077438059999999998 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.547941+0000", 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8725000000000001e-05 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.547948+0000", 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1339999999999998e-06 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.548016+0000", 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8096000000000007e-05 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.548176+0000", 2024-09-11T09:29:02.503 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00015980000000000001 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.548546+0000", 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00037019499999999998 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.555852+0000", 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0073066570000000003 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.558007+0000", 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.002155003 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.558073+0000", 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5745000000000002e-05 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.558129+0000", 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6205000000000001e-05 2024-09-11T09:29:02.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2466 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 950272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621700+0000", 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.853941387, 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013577165, 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2466 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621700+0000", 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621700+0000", 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621701+0000", 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1600000000000002e-07 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621703+0000", 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621709+0000", 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3910000000000003e-06 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621718+0000", 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6109999999999994e-06 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635088+0000", 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013370149 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635188+0000", 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010028599999999999 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635277+0000", 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9081999999999999e-05 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.510 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2477 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1605632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621724+0000", 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85391726599999995, 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013667881, 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2477 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621724+0000", 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621724+0000", 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621725+0000", 2024-09-11T09:29:02.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2099999999999996e-07 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621727+0000", 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3070000000000001e-06 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621734+0000", 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9970000000000001e-06 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621739+0000", 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.203e-06 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635309+0000", 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013569546 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635352+0000", 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2830999999999997e-05 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.515 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.515 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635392+0000", 2024-09-11T09:29:02.515 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0376000000000001e-05 2024-09-11T09:29:02.515 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2478 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1392640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621753+0000", 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85388834499999999, 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013853468000000001, 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.516 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2478 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621753+0000", 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621753+0000", 2024-09-11T09:29:02.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621754+0000", 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2100000000000002e-07 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621756+0000", 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3580000000000001e-06 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621763+0000", 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0310000000000001e-06 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621768+0000", 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2599999999999996e-06 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635433+0000", 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013665045000000001 2024-09-11T09:29:02.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635529+0000", 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5292999999999997e-05 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635607+0000", 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8059999999999995e-05 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2494 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1507328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621775+0000", 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85386664700000003, 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013961049999999999, 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2494 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.521 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621775+0000", 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621775+0000", 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621775+0000", 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.46e-07 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621777+0000", 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0090000000000002e-06 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621783+0000", 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0299999999999999e-06 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621788+0000", 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8409999999999999e-06 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635637+0000", 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013848974999999999 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635678+0000", 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1322999999999997e-05 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635736+0000", 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7426000000000002e-05 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.525 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2499 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621794+0000", 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.853847578, 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014072403000000001, 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2499 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.526 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621794+0000", 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621794+0000", 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621794+0000", 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621797+0000", 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.198e-06 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621803+0000", 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2480000000000001e-06 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621808+0000", 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.899e-06 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635768+0000", 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013960297999999999 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635815+0000", 2024-09-11T09:29:02.529 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7358999999999998e-05 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635866+0000", 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1010000000000001e-05 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2512 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621814+0000", 2024-09-11T09:29:02.530 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85382794799999995, 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014172888999999999, 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2512 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621814+0000", 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621814+0000", 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621814+0000", 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3799999999999998e-07 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621816+0000", 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7540000000000001e-06 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621821+0000", 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6640000000000003e-06 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621826+0000", 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9840000000000001e-06 2024-09-11T09:29:02.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635897+0000", 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014070332 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635944+0000", 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7092999999999998e-05 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635986+0000", 2024-09-11T09:29:02.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2724e-05 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2513 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1622016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621832+0000", 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.853809029, 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014271249, 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2513 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621832+0000", 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621832+0000", 2024-09-11T09:29:02.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621833+0000", 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5699999999999998e-07 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621835+0000", 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0229999999999999e-06 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621841+0000", 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0000000000000002e-06 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621845+0000", 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6299999999999997e-06 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636016+0000", 2024-09-11T09:29:02.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014170873 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636062+0000", 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5609999999999999e-05 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636104+0000", 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1755999999999998e-05 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2525 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1523712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621852+0000", 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85378907699999995, 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014369257999999999, 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2525 2024-09-11T09:29:02.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621852+0000", 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621852+0000", 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621860+0000", 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2849999999999997e-06 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621863+0000", 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9270000000000001e-06 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621870+0000", 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1069999999999999e-06 2024-09-11T09:29:02.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621875+0000", 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.31e-06 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636134+0000", 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014258533 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636180+0000", 2024-09-11T09:29:02.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5992999999999998e-05 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636222+0000", 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2103e-05 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2526 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1736704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621882+0000", 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85375922699999995, 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01445827, 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2526 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621882+0000", 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621882+0000", 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621883+0000", 2024-09-11T09:29:02.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0499999999999999e-07 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621885+0000", 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.046e-06 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621895+0000", 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0404000000000001e-05 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621900+0000", 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0719999999999999e-06 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636250+0000", 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014349883000000001 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636297+0000", 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7136999999999999e-05 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636341+0000", 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3322999999999998e-05 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2534 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1753088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621907+0000", 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85373495700000002, 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01454712, 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2534 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621907+0000", 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621907+0000", 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621907+0000", 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5499999999999999e-07 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621909+0000", 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.154e-06 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621915+0000", 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7869999999999998e-06 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621919+0000", 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6229999999999996e-06 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636369+0000", 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01444957 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636413+0000", 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4435999999999999e-05 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636454+0000", 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0194999999999997e-05 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2542 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1769472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621926+0000", 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85371575600000005, 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014655743000000001, 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2542 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.554 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621926+0000", 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621926+0000", 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621926+0000", 2024-09-11T09:29:02.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.46e-07 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621933+0000", 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9369999999999998e-06 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621939+0000", 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4330000000000001e-06 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.556 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621944+0000", 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6829999999999999e-06 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636483+0000", 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014538815 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636529+0000", 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6140999999999998e-05 2024-09-11T09:29:02.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636581+0000", 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2388000000000001e-05 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2545 3.1f 3:fca8606d:::rbd_data.1106b949ffb0.0000000000000002:7 [read 1785856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621970+0000", 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.85367116099999996, 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014720499, 2024-09-11T09:29:02.558 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2545 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621970+0000", 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621970+0000", 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621971+0000", 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2e-07 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621973+0000", 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621984+0000", 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0539e-05 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621989+0000", 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2800000000000003e-06 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636604+0000", 2024-09-11T09:29:02.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014615329999999999 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636650+0000", 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5952000000000003e-05 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.636691+0000", 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0568000000000002e-05 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.80446028000000003, 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.563 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:02.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:02.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2597 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [read 4096~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.679022+0000", 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.79661917800000004, 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014680222, 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2597 2024-09-11T09:29:02.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679022+0000", 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679022+0000", 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679024+0000", 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6530000000000001e-06 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679028+0000", 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6959999999999998e-06 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679035+0000", 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8929999999999999e-06 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679041+0000", 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0170000000000002e-06 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.687609+0000", 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085681279999999995 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.687672+0000", 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3764999999999997e-05 2024-09-11T09:29:02.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693500+0000", 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0058276300000000003 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693546+0000", 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6164e-05 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:02.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693703+0000", 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00015627600000000001 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:02.574 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:02.575 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:02.577 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:02.576+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:02.677 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:02.677+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:02.710 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:02.711 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:02.712 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:02.713 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:02.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-11T09:29:02.726 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:02.726 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:02.726 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:02.726 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:02.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:02.778 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:02.778+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:02.828 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone6 from 0xb605400 (overlap 0x6cec00) to 0x2b9000 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:02.868 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:02.869 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:02.871 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:02.871 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:02.878 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:02.879+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:02.888 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:02.888 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:02.888 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:02.888 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:02.888 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:02.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:02.979 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:02.980+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:03.046 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:03.046 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:03.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:03.046 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:03.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:382 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 13598720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.474948135, 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013989850999999999, 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 382 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571022+0000", 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2e-07 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571024+0000", 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3209999999999999e-06 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571031+0000", 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571036+0000", 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1749999999999996e-06 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584791+0000", 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013754377 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.584896+0000", 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105188 2024-09-11T09:29:03.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585011+0000", 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000115741 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:383 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 13860864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:29:03.051 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.4749270879999994, 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014215478, 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 383 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571043+0000", 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571053+0000", 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4579999999999999e-06 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571060+0000", 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9829999999999999e-06 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571066+0000", 2024-09-11T09:29:03.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1240000000000002e-06 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585059+0000", 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013993798 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585157+0000", 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7528999999999994e-05 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585258+0000", 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101104 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4530.0:384 3.8 3:1725d9a7:::rbd_data.11064c3f4ab3.0000000000000005:4 [read 14123008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:54.571074+0000", 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.4748957100000002, 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014425392, 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.056 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4530", 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1108066434", 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 384 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571074+0000", 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571074+0000", 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571075+0000", 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8400000000000005e-07 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571077+0000", 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1600000000000001e-06 2024-09-11T09:29:03.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571083+0000", 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3300000000000004e-06 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.571088+0000", 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.78e-06 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585291+0000", 2024-09-11T09:29:03.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014203386 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585386+0000", 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4495999999999998e-05 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:54.585499+0000", 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113756 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1411 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508954+0000", 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.537016156, 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013932020999999999, 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.061 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1411 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508954+0000", 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508954+0000", 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508954+0000", 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508956+0000", 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2749999999999998e-06 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508963+0000", 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5989999999999998e-06 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508968+0000", 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7999999999999998e-06 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522773+0000", 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013805105 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522815+0000", 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2160000000000003e-05 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522886+0000", 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0672000000000004e-05 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1413 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 606208~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508974+0000", 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5369958879999999, 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014056707, 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1413 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508974+0000", 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508974+0000", 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508974+0000", 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508976+0000", 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8899999999999999e-06 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508982+0000", 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8649999999999998e-06 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508986+0000", 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4839999999999998e-06 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522920+0000", 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013933157 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.522976+0000", 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6799999999999998e-05 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523031+0000", 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4135000000000003e-05 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1415 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 819200~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.069 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.508992+0000", 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.536977389, 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014169661, 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1415 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508992+0000", 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508992+0000", 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508993+0000", 2024-09-11T09:29:03.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.508995+0000", 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9939999999999999e-06 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509001+0000", 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1349999999999997e-06 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509005+0000", 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6179999999999997e-06 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523053+0000", 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01404802 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523107+0000", 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3523999999999998e-05 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523162+0000", 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4997999999999998e-05 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1425 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1261568~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509011+0000", 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5369585940000001, 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014272784, 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.074 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1425 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509011+0000", 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509011+0000", 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509012+0000", 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6600000000000002e-07 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509014+0000", 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6539999999999999e-06 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509020+0000", 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3260000000000001e-06 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509025+0000", 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5090000000000004e-06 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523184+0000", 2024-09-11T09:29:03.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014158949000000001 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523228+0000", 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4131999999999997e-05 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523284+0000", 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5847999999999998e-05 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1432 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 720896~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509033+0000", 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.53693685, 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01448846, 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1432 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509033+0000", 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509033+0000", 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509033+0000", 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.51e-07 2024-09-11T09:29:03.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509036+0000", 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1660000000000001e-06 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509042+0000", 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1319999999999999e-06 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509047+0000", 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8230000000000002e-06 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523326+0000", 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014279557 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523417+0000", 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0773000000000007e-05 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523521+0000", 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010445799999999999 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1441 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1376256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509053+0000", 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5369170030000001, 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014587788000000001, 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.083 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1441 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509053+0000", 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509053+0000", 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509053+0000", 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7399999999999999e-07 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509055+0000", 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.872e-06 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509061+0000", 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9649999999999996e-06 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509066+0000", 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6650000000000002e-06 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523548+0000", 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014481996 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523588+0000", 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0737e-05 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523641+0000", 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179000000000003e-05 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1445 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 835584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509072+0000", 2024-09-11T09:29:03.087 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5368979700000001, 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014792303, 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1445 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.088 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509072+0000", 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509072+0000", 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509072+0000", 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6600000000000002e-07 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509074+0000", 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9810000000000002e-06 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509080+0000", 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7509999999999997e-06 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509084+0000", 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6129999999999997e-06 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523674+0000", 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014589649999999999 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523759+0000", 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5237999999999999e-05 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523864+0000", 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010470400000000001 2024-09-11T09:29:03.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1457 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509112+0000", 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5368574420000001, 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014871565, 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1457 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509112+0000", 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509112+0000", 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509113+0000", 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509122+0000", 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7739999999999999e-06 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509128+0000", 2024-09-11T09:29:03.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5890000000000002e-06 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509133+0000", 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8389999999999997e-06 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523891+0000", 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014758405 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523933+0000", 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1452999999999998e-05 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.523984+0000", 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1125000000000003e-05 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1458 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 737280~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.096 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509141+0000", 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5368287299999999, 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015059629999999999, 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1458 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509141+0000", 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509141+0000", 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509141+0000", 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5999999999999999e-07 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509143+0000", 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.486e-06 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509147+0000", 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7160000000000001e-06 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509151+0000", 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7979999999999996e-06 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524019+0000", 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.014867818 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524103+0000", 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3825999999999993e-05 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524201+0000", 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7626000000000003e-05 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.100 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1467 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509157+0000", 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.536812351, 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015162072, 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1467 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509157+0000", 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509157+0000", 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509158+0000", 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509159+0000", 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3239999999999999e-06 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509163+0000", 2024-09-11T09:29:03.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7510000000000002e-06 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509165+0000", 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.593e-06 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524226+0000", 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015060272 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524268+0000", 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2435000000000003e-05 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524319+0000", 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1455000000000001e-05 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.105 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1473 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509169+0000", 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.536800288, 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015364241000000001, 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.106 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1473 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509169+0000", 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509169+0000", 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509170+0000", 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.28e-07 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509171+0000", 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.085e-06 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509174+0000", 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0759999999999999e-06 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509176+0000", 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4080000000000001e-06 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524354+0000", 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015178043 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524437+0000", 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.263e-05 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524534+0000", 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6770999999999999e-05 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1475 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509179+0000", 2024-09-11T09:29:03.110 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5367904509999999, 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015475932, 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1475 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509179+0000", 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509179+0000", 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509180+0000", 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3300000000000001e-07 2024-09-11T09:29:03.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509181+0000", 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.249e-06 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509184+0000", 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.027e-06 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509186+0000", 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6529999999999998e-06 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524559+0000", 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01537296 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524601+0000", 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2021999999999998e-05 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524655+0000", 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3788000000000001e-05 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1484 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1523712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509191+0000", 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.536778964, 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015681869000000001, 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1484 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509191+0000", 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509191+0000", 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509191+0000", 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9400000000000001e-07 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509193+0000", 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.88e-06 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509199+0000", 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6350000000000003e-06 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509203+0000", 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3499999999999999e-06 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.118 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524685+0000", 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015482477 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524769+0000", 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3737000000000001e-05 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524873+0000", 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103496 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1501 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1474560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509210+0000", 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.53676009, 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015784550000000001, 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1501 2024-09-11T09:29:03.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509210+0000", 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509210+0000", 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509210+0000", 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8799999999999998e-07 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509212+0000", 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0329999999999998e-06 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509219+0000", 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7769999999999997e-06 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509224+0000", 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7129999999999996e-06 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524899+0000", 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015675687000000001 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524940+0000", 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0844999999999999e-05 2024-09-11T09:29:03.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.524994+0000", 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4107000000000002e-05 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1509 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509253+0000", 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.53671723, 2024-09-11T09:29:03.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015957024, 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1509 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509253+0000", 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509253+0000", 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509253+0000", 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7e-07 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509256+0000", 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4659999999999999e-06 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509265+0000", 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2930000000000001e-06 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509271+0000", 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0850000000000002e-06 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.525028+0000", 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015757611000000001 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.525113+0000", 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4401999999999996e-05 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.525210+0000", 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6696999999999999e-05 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.128 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:1513 3.8 3:1346800d:::rbd_data.1106b949ffb0.000000000000000b:7 [read 1605632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.509280+0000", 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.5366892999999999, 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016035589999999999, 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1513 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509280+0000", 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509280+0000", 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509281+0000", 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509283+0000", 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2290000000000002e-06 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509291+0000", 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.323e-06 2024-09-11T09:29:03.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.509296+0000", 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4589999999999997e-06 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.525226+0000", 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015929888 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.525267+0000", 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0828000000000003e-05 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.525316+0000", 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9345999999999998e-05 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2551 3.8 3:1682ee56:::rbd_data.1106b949ffb0.0000000000000000:7 [read 1818624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:03.133 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.621974+0000", 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.423995742, 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013862381999999999, 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2551 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621974+0000", 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621974+0000", 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621974+0000", 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9900000000000001e-07 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621983+0000", 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5669999999999995e-06 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621990+0000", 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0249999999999997e-06 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.621996+0000", 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.614e-06 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635721+0000", 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013725759000000001 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635781+0000", 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-05 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.635836+0000", 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5717999999999997e-05 2024-09-11T09:29:03.137 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.138 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:03.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:03.140 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:03.140+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:03.187 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-11T09:29:03.240 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:03.241+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:925 write 0x9551600 thru 0x955e1cf (0xcbd0 bytes) 2024-09-11T09:29:03.268 INFO:teuthology.orchestra.run.smithi107.stdout:926 trunc from 0xb605400 to 0x9700400 2024-09-11T09:29:03.290 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:03.290 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:03.291 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:03.291 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:03.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:03.340 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:03.341+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:03.351 INFO:teuthology.orchestra.run.smithi107.stdout:927 read 0x7fb7600 thru 0x7fc147c (0x9e7d bytes) 2024-09-11T09:29:03.354 INFO:teuthology.orchestra.run.smithi107.stdout:929 write 0x8506e00 thru 0x850b327 (0x4528 bytes) 2024-09-11T09:29:03.355 INFO:teuthology.orchestra.run.smithi107.stdout:931 read 0x46c7800 thru 0x46cd18e (0x598f bytes) 2024-09-11T09:29:03.357 INFO:teuthology.orchestra.run.smithi107.stdout:933 read 0x42f7a00 thru 0x42f9d11 (0x2312 bytes) 2024-09-11T09:29:03.358 INFO:teuthology.orchestra.run.smithi107.stdout:938 read 0x19af200 thru 0x19b4807 (0x5608 bytes) 2024-09-11T09:29:03.358 INFO:teuthology.orchestra.run.smithi107.stdout:939 read 0x6286000 thru 0x6294e76 (0xee77 bytes) 2024-09-11T09:29:03.359 INFO:teuthology.orchestra.run.smithi107.stdout:940 trunc from 0x9700400 to 0x93db600 2024-09-11T09:29:03.418 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:03.418 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:03.418 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:03.418 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:03.418 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:03.427 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:03.429 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:03.471 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:03.472+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:03.537 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:03.537 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:03.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:03.537 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:03.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.07786647, 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:03.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:03.542 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.816346782, 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:03.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:03.548 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.815100814999999, 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:03.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:03.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:03.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:03.553 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:03.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:03.555 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:03.555+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:03.598 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:03.598 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:03.612 INFO:teuthology.orchestra.run.smithi107.stdout:941 read 0x4e88c00 thru 0x4e914b7 (0x88b8 bytes) 2024-09-11T09:29:03.616 INFO:teuthology.orchestra.run.smithi107.stdout:942 write 0x6a04c00 thru 0x6a062bd (0x16be bytes) 2024-09-11T09:29:03.616 INFO:teuthology.orchestra.run.smithi107.stdout:946 punch from 0x3410000 to 0x341cc00, (0xcc00 bytes) 2024-09-11T09:29:03.617 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:03.618 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:03.626 INFO:teuthology.orchestra.run.smithi107.stdout:950 write 0x2a7b400 thru 0x2a86e66 (0xba67 bytes) 2024-09-11T09:29:03.626 INFO:teuthology.orchestra.run.smithi107.stdout:951 read 0x2172600 thru 0x2176659 (0x405a bytes) 2024-09-11T09:29:03.631 INFO:teuthology.orchestra.run.smithi107.stdout:952 trunc from 0x93db600 to 0xdb65c00 2024-09-11T09:29:03.637 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:03.638 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:03.655 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:03.655+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:03.660 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:03.660 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:953 write 0x5479a00 thru 0x547dfd3 (0x45d4 bytes) 2024-09-11T09:29:03.673 INFO:teuthology.orchestra.run.smithi107.stdout:954 write 0xaf77c00 thru 0xaf7af88 (0x3389 bytes) 2024-09-11T09:29:03.677 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:03.682 INFO:teuthology.orchestra.run.smithi107.stdout:957 trunc from 0xdb65c00 to 0x58b6200 2024-09-11T09:29:03.691 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:03.691 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:03.702 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:03.704 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:03.704 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:03.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:29:03.787 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:03.788+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:03.857 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:03.857+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:03.859 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:03.859 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:03.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:03.957 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:03.957+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:04.013 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.013 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.022 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:04.062 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:04.063+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:04.149 INFO:teuthology.orchestra.run.smithi107.stdout:958 punch from 0x1762200 to 0x1766200, (0x4000 bytes) 2024-09-11T09:29:04.158 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:04.159+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:04.172 INFO:teuthology.orchestra.run.smithi107.stdout:959 write 0x7130e00 thru 0x71406da (0xf8db bytes) 2024-09-11T09:29:04.176 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.176 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.182 INFO:teuthology.orchestra.run.smithi107.stdout:962 trunc from 0x7140800 to 0xa524e00 2024-09-11T09:29:04.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:04.226 INFO:teuthology.orchestra.run.smithi107.stdout:963 write 0xd45fa00 thru 0xd46875a (0x8d5b bytes) 2024-09-11T09:29:04.233 INFO:teuthology.orchestra.run.smithi107.stdout:964 read 0xade4600 thru 0xadef20f (0xac10 bytes) 2024-09-11T09:29:04.242 INFO:teuthology.orchestra.run.smithi107.stdout:966 trunc from 0xd468800 to 0x5798a00 2024-09-11T09:29:04.259 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:04.259+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:04.292 INFO:teuthology.orchestra.run.smithi107.stdout:967 read 0x4124e00 thru 0x4133c34 (0xee35 bytes) 2024-09-11T09:29:04.312 INFO:teuthology.orchestra.run.smithi107.stdout:969 write 0x9d12600 thru 0x9d208a8 (0xe2a9 bytes) 2024-09-11T09:29:04.316 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.316 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.323 INFO:teuthology.orchestra.run.smithi107.stdout:970 trunc from 0x9d20a00 to 0x3b3e00 2024-09-11T09:29:04.325 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:04.360 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:04.361+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:04.462 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:04.462+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:04.504 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.504 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:04.562 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:04.563+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:04.633 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.633 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:04.663 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:04.664+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:04.694 INFO:teuthology.orchestra.run.smithi107.stdout:971 clone 8 order 22 su 32768 sc 9 2024-09-11T09:29:04.752 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:29:04.764 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:04.764+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:04.794 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.794 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.803 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:04.842 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone7 from 0x3b3e00 (overlap 0x3b3e00) to 0x5ec00 2024-09-11T09:29:04.864 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:04.865+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:04.915 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:04.915 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:04.923 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:04.967 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:04.968+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:05.031 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:05.031 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:05.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:05.067 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:05.067+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:05.168 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:05.169+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:05.186 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:05.186 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:05.195 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:05.269 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:05.269+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:05.302 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:05.302 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:05.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:05.373 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:05.373+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:05.420 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:05.420 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:05.429 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:05.445 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-11T09:29:05.513 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:05.514+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:05.517 INFO:teuthology.orchestra.run.smithi107.stdout:975 read 0x1c8c00 thru 0x1d5020 (0xc421 bytes) 2024-09-11T09:29:05.520 INFO:teuthology.orchestra.run.smithi107.stdout:976 punch from 0x281000 to 0x287200, (0x6200 bytes) 2024-09-11T09:29:05.542 INFO:teuthology.orchestra.run.smithi107.stdout:980 trunc from 0x3b3e00 to 0x4895800 2024-09-11T09:29:05.557 INFO:teuthology.orchestra.run.smithi107.stdout:981 write 0x11a4c00 thru 0x11a5ce7 (0x10e8 bytes) 2024-09-11T09:29:05.557 INFO:teuthology.orchestra.run.smithi107.stdout:982 read 0x5ea00 thru 0x6b597 (0xcb98 bytes) 2024-09-11T09:29:05.559 INFO:teuthology.orchestra.run.smithi107.stdout:983 clone 9 order 20 su 8192 sc 4 2024-09-11T09:29:05.577 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:05.577+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:05.586 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:05.586 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:05.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:05.679 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:05.680+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:05.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:05.743 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:05.743 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:05.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:05.795 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:05.795+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:05.851 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:05.852 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:05.860 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone8 from 0x4895800 (overlap 0x3b3e00) to 0x297800 2024-09-11T09:29:05.877 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:05.881 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:05.882+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:05.911 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:05.911 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:05.924 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:05.925 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:05.925 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:05.929 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:05.929 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:05.934 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:05.946 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:05.963 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:05.964 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:05.982 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:05.983+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:06.047 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:06.047 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:06.047 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:06.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:06.099 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:06.100+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:06.163 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:06.163 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 29.704290887999999, 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:06.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:06.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:06.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:06.168 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.442771199999999, 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:06.169 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:06.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:06.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:06.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 17.441525233, 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:06.175 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:06.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:06.180 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:06.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:06.184 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:06.184+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:06.284 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:06.284 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:06.284 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:06.284 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:06.285 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:06.286+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:06.292 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:06.386 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:06.386+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:06.393 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:06.393 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:06.394 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:06.394 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:06.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:06.486 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:06.487+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:06.547 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:06.547 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.109837783000003, 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:06.548 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:06.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:06.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3812 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2246656~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.518026+0000", 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.028810896, 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042738125000000002, 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3812 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518026+0000", 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518026+0000", 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518027+0000", 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518028+0000", 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.765e-06 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518036+0000", 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7049999999999999e-06 2024-09-11T09:29:06.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518041+0000", 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5959999999999997e-06 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.560621+0000", 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042580673999999999 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.560697+0000", 2024-09-11T09:29:06.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5327999999999994e-05 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.560764+0000", 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7695999999999997e-05 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3813 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2273280~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.518047+0000", 2024-09-11T09:29:06.559 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0287904000000001, 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042935208000000002, 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3813 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518047+0000", 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518047+0000", 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518047+0000", 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7e-07 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518049+0000", 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518055+0000", 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8579999999999996e-06 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518060+0000", 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7879999999999997e-06 2024-09-11T09:29:06.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.560807+0000", 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042746696000000001 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.560899+0000", 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2672999999999999e-05 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.560982+0000", 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2708999999999997e-05 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3814 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2314240~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.518066+0000", 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0287714490000002, 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121746000000002, 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3814 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518066+0000", 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518066+0000", 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518066+0000", 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8299999999999998e-07 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518068+0000", 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0140000000000001e-06 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518074+0000", 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8549999999999998e-06 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518079+0000", 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7450000000000003e-06 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.561027+0000", 2024-09-11T09:29:06.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042948275000000001 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.561111+0000", 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4291999999999999e-05 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.561188+0000", 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6181999999999997e-05 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3815 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2340864~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.518084+0000", 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0287530469999999, 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04328133, 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3815 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518084+0000", 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.518084+0000", 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.519868+0000", 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001783312 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.519872+0000", 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4769999999999997e-06 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.519881+0000", 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3710000000000006e-06 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.519888+0000", 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.561211+0000", 2024-09-11T09:29:06.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041322991000000003 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.561294+0000", 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3072999999999996e-05 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.561366+0000", 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1606999999999994e-05 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3823 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2557952~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520090+0000", 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0267467969999999, 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042680056000000001, 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3823 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520090+0000", 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520090+0000", 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520091+0000", 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520093+0000", 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2709999999999999e-06 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520099+0000", 2024-09-11T09:29:06.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0020000000000003e-06 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520104+0000", 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562632+0000", 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042527464000000001 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562686+0000", 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4271000000000003e-05 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562771+0000", 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4569999999999998e-05 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3824 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2584576~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520110+0000", 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0267268189999998, 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042851405000000002, 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3824 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520110+0000", 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520110+0000", 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520111+0000", 2024-09-11T09:29:06.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-07 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520113+0000", 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1299999999999999e-06 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520119+0000", 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8690000000000001e-06 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520124+0000", 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0509999999999996e-06 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562809+0000", 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042685386999999998 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562893+0000", 2024-09-11T09:29:06.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3356000000000006e-05 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562962+0000", 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9222999999999998e-05 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3825 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2611200~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520130+0000", 2024-09-11T09:29:06.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0267073130000002, 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043001683999999998, 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3825 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520130+0000", 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520130+0000", 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520130+0000", 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520132+0000", 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0889999999999998e-06 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520138+0000", 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0619999999999997e-06 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520143+0000", 2024-09-11T09:29:06.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9219999999999996e-06 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.562993+0000", 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042849960999999999 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563049+0000", 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5671999999999998e-05 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563132+0000", 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2596999999999995e-05 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3826 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2625536~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520157+0000", 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0266798920000002, 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043156967999999997, 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3826 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520157+0000", 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520157+0000", 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520158+0000", 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.75e-07 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520160+0000", 2024-09-11T09:29:06.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.328e-06 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520166+0000", 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.207e-06 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520172+0000", 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4929999999999998e-06 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563168+0000", 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042995959 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563244+0000", 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6452999999999995e-05 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563314+0000", 2024-09-11T09:29:06.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9952999999999999e-05 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3827 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2652160~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520178+0000", 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0266589499999998, 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043306274999999998, 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3827 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520178+0000", 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520178+0000", 2024-09-11T09:29:06.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520179+0000", 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2799999999999996e-07 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520181+0000", 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.119e-06 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520187+0000", 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8889999999999999e-06 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520191+0000", 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6310000000000002e-06 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563344+0000", 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043152446999999997 2024-09-11T09:29:06.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563402+0000", 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7921000000000003e-05 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563485+0000", 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2739999999999997e-05 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3828 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2678784~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520199+0000", 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0266386889999999, 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466527999999997, 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3828 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520199+0000", 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520199+0000", 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520199+0000", 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520201+0000", 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1380000000000001e-06 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520207+0000", 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1090000000000003e-06 2024-09-11T09:29:06.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520212+0000", 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6539999999999998e-06 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563522+0000", 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043309813000000003 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563596+0000", 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4037999999999998e-05 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563665+0000", 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9388999999999996e-05 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3829 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2693120~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.520218+0000", 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0266192099999998, 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580346999999998, 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3829 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520218+0000", 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.520218+0000", 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.521865+0000", 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001647035 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.521869+0000", 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6820000000000001e-06 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.521878+0000", 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0790000000000007e-06 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.521884+0000", 2024-09-11T09:29:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1120000000000001e-06 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563701+0000", 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041817144000000001 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563752+0000", 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.563798+0000", 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6888999999999998e-05 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3835 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2828288~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.522004+0000", 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0248328839999998, 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042756111999999999, 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3835 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522004+0000", 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522004+0000", 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522005+0000", 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6699999999999999e-07 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522007+0000", 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.142e-06 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522013+0000", 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0789999999999997e-06 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522018+0000", 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8180000000000003e-06 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564627+0000", 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042609213 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564680+0000", 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3445000000000001e-05 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564760+0000", 2024-09-11T09:29:06.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0048000000000003e-05 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3836 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2854912~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.522024+0000", 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0248137289999999, 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042890685999999997, 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3836 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522024+0000", 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522024+0000", 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522024+0000", 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4900000000000001e-07 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522026+0000", 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522032+0000", 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522037+0000", 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8409999999999999e-06 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564797+0000", 2024-09-11T09:29:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042760409999999999 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564875+0000", 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7872000000000003e-05 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564914+0000", 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9233999999999997e-05 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3837 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2881536~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.522056+0000", 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0247817119999998, 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067197000000002, 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3837 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522056+0000", 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522056+0000", 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522056+0000", 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522058+0000", 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3020000000000002e-06 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522065+0000", 2024-09-11T09:29:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2230000000000004e-06 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522071+0000", 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8560000000000001e-06 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.564958+0000", 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042886279999999999 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565043+0000", 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5730999999999996e-05 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565123+0000", 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9313000000000004e-05 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3838 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2895872~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.522079+0000", 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0247578289999999, 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043145065000000003, 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3838 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522079+0000", 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522079+0000", 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522080+0000", 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522082+0000", 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1830000000000001e-06 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522090+0000", 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3259999999999996e-06 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522095+0000", 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565139+0000", 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043043831999999997 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565185+0000", 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5865999999999998e-05 2024-09-11T09:29:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565224+0000", 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9416999999999999e-05 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3839 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2922496~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.522103+0000", 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0247345779999999, 2024-09-11T09:29:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043238956000000002, 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3839 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522103+0000", 2024-09-11T09:29:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522103+0000", 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522103+0000", 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7800000000000002e-07 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522105+0000", 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1330000000000002e-06 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522111+0000", 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1770000000000003e-06 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522119+0000", 2024-09-11T09:29:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1380000000000001e-06 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565255+0000", 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043136051000000002 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565299+0000", 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4855999999999999e-05 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565342+0000", 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2222999999999999e-05 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3840 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 2963456~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.522127+0000", 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0247106430000001, 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043324121, 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3840 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522127+0000", 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.522127+0000", 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.523867+0000", 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001740147 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.523871+0000", 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3189999999999996e-06 2024-09-11T09:29:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.523881+0000", 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0091999999999999e-05 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.523888+0000", 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565367+0000", 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041478741 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565413+0000", 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6263000000000001e-05 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.565451+0000", 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7620000000000002e-05 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3851 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 3328000~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.524144+0000", 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0226929400000002, 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042713005999999998, 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3851 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524144+0000", 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524144+0000", 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524145+0000", 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-07 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524147+0000", 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2670000000000001e-06 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524154+0000", 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5010000000000001e-06 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524160+0000", 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2700000000000001e-06 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.566692+0000", 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042532545999999997 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.566777+0000", 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4841999999999998e-05 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.566857+0000", 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0067000000000004e-05 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:3852 3.13 3:c9827273:::rbd_data.1106259a042c.0000000000000011:9 [read 3342336~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.524168+0000", 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.0226696679999998, 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042908393000000003, 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 3852 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524168+0000", 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.524168+0000", 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.525866+0000", 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0016980859999999999 2024-09-11T09:29:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.525870+0000", 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0199999999999996e-06 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.525879+0000", 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2469999999999997e-06 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.525885+0000", 2024-09-11T09:29:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.566907+0000", 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.041022212000000002 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.566999+0000", 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1226999999999999e-05 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.567076+0000", 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7319999999999998e-05 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:06.645 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:06.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:06.647 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:06.647+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:06.747 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:06.747+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:06.766 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:29:06.800 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:06.800 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:06.801 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:06.801 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:06.809 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:06.848 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:06.848+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:06.922 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-11T09:29:06.949 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:06.949+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:06.956 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:06.957 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:06.957 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:06.957 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:06.957 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:06.965 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:07.011 INFO:teuthology.orchestra.run.smithi107.stdout:984 write 0xec00c00 thru 0xec01593 (0x994 bytes) 2024-09-11T09:29:07.012 INFO:teuthology.orchestra.run.smithi107.stdout:986 punch from 0x4972600 to 0x497e400, (0xbe00 bytes) 2024-09-11T09:29:07.013 INFO:teuthology.orchestra.run.smithi107.stdout:987 read 0x3d06400 thru 0x3d0920f (0x2e10 bytes) 2024-09-11T09:29:07.013 INFO:teuthology.orchestra.run.smithi107.stdout:988 write 0x49f2c00 thru 0x49f8f9e (0x639f bytes) 2024-09-11T09:29:07.013 INFO:teuthology.orchestra.run.smithi107.stdout:989 read 0x6a7ce00 thru 0x6a839db (0x6bdc bytes) 2024-09-11T09:29:07.013 INFO:teuthology.orchestra.run.smithi107.stdout:990 write 0xebc2200 thru 0xebc3cf7 (0x1af8 bytes) 2024-09-11T09:29:07.015 INFO:teuthology.orchestra.run.smithi107.stdout:991 trunc from 0xec01600 to 0xccda600 2024-09-11T09:29:07.042 INFO:teuthology.orchestra.run.smithi107.stdout:992 read 0x4ee0800 thru 0x4ee8c31 (0x8432 bytes) 2024-09-11T09:29:07.049 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:07.050+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:07.058 INFO:teuthology.orchestra.run.smithi107.stdout:994 trunc from 0xccda600 to 0x3c7b000 2024-09-11T09:29:07.093 INFO:teuthology.orchestra.run.smithi107.stdout:996 write 0x80ec400 thru 0x80fab33 (0xe734 bytes) 2024-09-11T09:29:07.094 INFO:teuthology.orchestra.run.smithi107.stdout:997 clone 10 order 19 su 2048 sc 6 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7623772799999999, 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:07.130 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:07.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:07.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7623398950000002, 2024-09-11T09:29:07.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:07.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:07.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:07.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:07.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:07.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:07.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:07.142 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7623205770000001, 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.762309331, 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:07.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:07.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7622674260000002, 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:07.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:07.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:07.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:07.156 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7622500219999999, 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:07.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7622321269999999, 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:07.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:07.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7622119469999999, 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:07.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:07.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:07.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7621909630000001, 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:07.170 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:07.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7621638659999999, 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:07.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7621445790000001, 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:07.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:07.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.761940219, 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:07.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:07.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7614064630000001, 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:07.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:07.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:07.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7613575699999999, 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:07.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:07.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7613094820000001, 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:07.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7612714440000001, 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:07.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:07.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7610476020000001, 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:07.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.214 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7608705000000002, 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:07.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7607249449999998, 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:07.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:07.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 3.7606035589999998, 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:07.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:07.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:07.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.228 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:07.230 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:07.230 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:07.230+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:07.330 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:07.331+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:07.373 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:07.373 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:07.373 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:07.382 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:07.465 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:07.466+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:07.524 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:07.524 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:07.525 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:07.525 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:07.525 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:07.532 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:07.531+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:07.533 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:07.634 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:07.634+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2479 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1605632~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632834+0000", 2024-09-11T09:29:07.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.043922706, 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028843035, 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2479 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632834+0000", 2024-09-11T09:29:07.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632834+0000", 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632835+0000", 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1999999999999999e-07 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632839+0000", 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9949999999999999e-06 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632852+0000", 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2809000000000001e-05 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632859+0000", 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3499999999999999e-06 2024-09-11T09:29:07.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661497+0000", 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028637493999999999 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661586+0000", 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9289999999999994e-05 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661677+0000", 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1377000000000003e-05 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2483 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1064960~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0438542479999997, 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029001923999999998, 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2483 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632903+0000", 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632910+0000", 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1069999999999999e-06 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632917+0000", 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9670000000000004e-06 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632922+0000", 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0429999999999998e-06 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661722+0000", 2024-09-11T09:29:07.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028799495000000001 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661813+0000", 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1118000000000004e-05 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661905+0000", 2024-09-11T09:29:07.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1679000000000001e-05 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2492 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.04380658, 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029187141, 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2492 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632951+0000", 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.96e-07 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632953+0000", 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0980000000000001e-06 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632959+0000", 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.212e-06 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632964+0000", 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8579999999999999e-06 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.661950+0000", 2024-09-11T09:29:07.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.028986369000000001 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662047+0000", 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6718999999999993e-05 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662138+0000", 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0488999999999994e-05 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2493 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1720320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632970+0000", 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0437866900000001, 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029393354999999999, 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2493 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632970+0000", 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632970+0000", 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632971+0000", 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632973+0000", 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2579999999999998e-06 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632979+0000", 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1890000000000004e-06 2024-09-11T09:29:07.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632985+0000", 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9209999999999997e-06 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662182+0000", 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029197330000000001 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662272+0000", 2024-09-11T09:29:07.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9272999999999997e-05 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662364+0000", 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1998999999999998e-05 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2496 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1507328~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0437654910000003, 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029611141000000001, 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2496 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632992+0000", 2024-09-11T09:29:07.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5600000000000001e-07 2024-09-11T09:29:07.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.632994+0000", 2024-09-11T09:29:07.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:29:07.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633000+0000", 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.643e-06 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633004+0000", 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5820000000000003e-06 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662409+0000", 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029404354000000001 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662510+0000", 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101547 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662603+0000", 2024-09-11T09:29:07.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2357999999999999e-05 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2497 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 966656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0437463669999998, 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029818801999999998, 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2497 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633011+0000", 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633013+0000", 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0990000000000001e-06 2024-09-11T09:29:07.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633019+0000", 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0170000000000002e-06 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633024+0000", 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6410000000000001e-06 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662645+0000", 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029621056 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662743+0000", 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8292e-05 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662830+0000", 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6324999999999998e-05 2024-09-11T09:29:07.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2500 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 966656~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0436613030000004, 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029839158000000001, 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2500 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:07.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633096+0000", 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.06e-07 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633099+0000", 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9069999999999999e-06 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633107+0000", 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7370000000000006e-06 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633113+0000", 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7400000000000001e-06 2024-09-11T09:29:07.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662847+0000", 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029734159999999999 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662896+0000", 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-05 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662935+0000", 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9258000000000003e-05 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2506 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1409024~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633141+0000", 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0436159099999998, 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030041090999999999, 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2506 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633141+0000", 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633141+0000", 2024-09-11T09:29:07.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633142+0000", 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.32e-07 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633144+0000", 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3479999999999998e-06 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633150+0000", 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633155+0000", 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7740000000000004e-06 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.662976+0000", 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029821070000000002 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663082+0000", 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105512 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663182+0000", 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010068699999999999 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2511 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1622016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633160+0000", 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0435969910000003, 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030109462, 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2511 2024-09-11T09:29:07.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633160+0000", 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633160+0000", 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633161+0000", 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2100000000000002e-07 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633167+0000", 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2260000000000002e-06 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633173+0000", 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2230000000000004e-06 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633178+0000", 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7589999999999997e-06 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663194+0000", 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030016602999999999 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663233+0000", 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9014999999999998e-05 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663270+0000", 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6214999999999998e-05 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.722 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2514 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1081344~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0435731940000004, 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030311439999999999, 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2514 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633184+0000", 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4400000000000001e-07 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633186+0000", 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633192+0000", 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0990000000000004e-06 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633197+0000", 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.634e-06 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663309+0000", 2024-09-11T09:29:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.03011229 2024-09-11T09:29:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663407+0000", 2024-09-11T09:29:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.7990000000000002e-05 2024-09-11T09:29:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663495+0000", 2024-09-11T09:29:07.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8140999999999995e-05 2024-09-11T09:29:07.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2516 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1622016~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0435502650000004, 2024-09-11T09:29:07.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030377168, 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2516 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633207+0000", 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4000000000000002e-07 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633210+0000", 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5600000000000001e-06 2024-09-11T09:29:07.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633217+0000", 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0489999999999998e-06 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633222+0000", 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4369999999999998e-06 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663508+0000", 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030285362 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663546+0000", 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8631999999999999e-05 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663584+0000", 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7688000000000002e-05 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2523 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1523712~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0434493260000002, 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030473742000000002, 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2523 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633308+0000", 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8299999999999998e-07 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633310+0000", 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633316+0000", 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633321+0000", 2024-09-11T09:29:07.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8590000000000004e-06 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663629+0000", 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030308348999999998 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663709+0000", 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0062999999999996e-05 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663782+0000", 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2169000000000006e-05 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2528 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1736704~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633327+0000", 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0434298130000004, 2024-09-11T09:29:07.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030671588999999999, 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2528 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633327+0000", 2024-09-11T09:29:07.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633327+0000", 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633328+0000", 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633330+0000", 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.165e-06 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633336+0000", 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0140000000000004e-06 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633340+0000", 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5900000000000001e-06 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663826+0000", 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030485760000000001 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663917+0000", 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0370000000000004e-05 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.663999+0000", 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2287999999999995e-05 2024-09-11T09:29:07.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2530 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1851392~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0434101570000003, 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030819236999999999, 2024-09-11T09:29:07.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2530 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633347+0000", 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.39e-07 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633349+0000", 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9630000000000001e-06 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633355+0000", 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5230000000000003e-06 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633359+0000", 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4889999999999997e-06 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664021+0000", 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030661702999999998 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664097+0000", 2024-09-11T09:29:07.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6320000000000001e-05 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664166+0000", 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8800000000000005e-05 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2533 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1753088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633385+0000", 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0433717270000002, 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030977601, 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2533 2024-09-11T09:29:07.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633385+0000", 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633385+0000", 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633386+0000", 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5999999999999999e-07 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633388+0000", 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.12e-06 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633394+0000", 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0800000000000002e-06 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633399+0000", 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7600000000000002e-06 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664204+0000", 2024-09-11T09:29:07.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030805018 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664286+0000", 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2503000000000006e-05 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664363+0000", 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6760000000000004e-05 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2536 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1753088~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633456+0000", 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.04330108, 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031090034999999999, 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2536 2024-09-11T09:29:07.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.758 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633456+0000", 2024-09-11T09:29:07.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633456+0000", 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633457+0000", 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7100000000000001e-07 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633460+0000", 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8700000000000001e-06 2024-09-11T09:29:07.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633467+0000", 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7789999999999996e-06 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633473+0000", 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1340000000000001e-06 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664398+0000", 2024-09-11T09:29:07.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.030924423999999999 2024-09-11T09:29:07.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664475+0000", 2024-09-11T09:29:07.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7418999999999999e-05 2024-09-11T09:29:07.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664546+0000", 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0737999999999999e-05 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2540 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0432765499999999, 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031258706999999997, 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2540 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633481+0000", 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633483+0000", 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1849999999999998e-06 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633490+0000", 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0419999999999997e-06 2024-09-11T09:29:07.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633496+0000", 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5709999999999997e-06 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664583+0000", 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031087571000000001 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664663+0000", 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9374000000000003e-05 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664739+0000", 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6538999999999993e-05 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2543 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1785856~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633503+0000", 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0432536209999999, 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031419903999999999, 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2543 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633503+0000", 2024-09-11T09:29:07.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633503+0000", 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633504+0000", 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3500000000000002e-07 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633511+0000", 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7510000000000002e-06 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633518+0000", 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9639999999999997e-06 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633523+0000", 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:07.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664774+0000", 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031250759000000003 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664850+0000", 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5644999999999998e-05 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664923+0000", 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3747999999999999e-05 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2544 3.10 3:0a37aeb9:::rbd_data.1106b949ffb0.0000000000000003:7 [read 1769472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0432252499999999, 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031595493000000002, 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2544 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633532+0000", 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633534+0000", 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1380000000000001e-06 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633541+0000", 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0550000000000002e-06 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633547+0000", 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2970000000000003e-06 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.664961+0000", 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031414624000000002 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665047+0000", 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5527999999999998e-05 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665127+0000", 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0390999999999997e-05 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4508.0:2548 3.10 3:0c9f0b00:::rbd_data.1106b949ffb0.0000000000000001:7 [read 1802240~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:01.633553+0000", 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.043203922, 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031761585000000002, 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:07.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4508", 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2548 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633553+0000", 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633553+0000", 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633554+0000", 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1199999999999998e-07 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:07.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633556+0000", 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.024e-06 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633562+0000", 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4629999999999998e-06 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.633567+0000", 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1000000000000003e-06 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665162+0000", 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.031594672999999997 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665243+0000", 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1167000000000004e-05 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:01.665315+0000", 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1746000000000001e-05 2024-09-11T09:29:07.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:07.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:07.785 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:07.787 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:29:07.787 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:07.785+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:07.787 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:07.873 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:07.873 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:07.873 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T09:29:07.873 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-11T09:29:07.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-11T09:29:07.883 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone9 from 0x80fac00 (overlap 0x3c7b000) to 0x2a56000 2024-09-11T09:29:07.887 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:07.887+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:07.957 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:07.957 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:07.957 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:07.957 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:07.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:07.980 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:08.010 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:08.011+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:08.089 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:08.090+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:08.093 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:08.093 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:08.093 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:08.093 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:08.093 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:08.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:08.152 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:08.152 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:08.170 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:08.171 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:08.187 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:08.187 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:08.190 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:08.190+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:08.206 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:08.206 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:08.226 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:08.254 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:08.254 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:08.266 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:08.266 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:08.276 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:08.291 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:08.291+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:08.364 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-11T09:29:08.392 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:08.392+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:08.425 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:08.426 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:08.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:08.438 INFO:teuthology.orchestra.run.smithi107.stdout:1000 read 0x5880400 thru 0x5886570 (0x6171 bytes) 2024-09-11T09:29:08.438 INFO:teuthology.orchestra.run.smithi107.stdout:1001 read 0x1980600 thru 0x1983407 (0x2e08 bytes) 2024-09-11T09:29:08.439 INFO:teuthology.orchestra.run.smithi107.stdout:1002 read 0x6b91e00 thru 0x6b92c88 (0xe89 bytes) 2024-09-11T09:29:08.439 INFO:teuthology.orchestra.run.smithi107.stdout:1003 trunc from 0x80fac00 to 0x77bce00 2024-09-11T09:29:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:1004 trunc from 0x77bce00 to 0x138a600 2024-09-11T09:29:08.493 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:08.492+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:08.543 INFO:teuthology.orchestra.run.smithi107.stdout:1005 read 0xb29000 thru 0xb3862b (0xf62c bytes) 2024-09-11T09:29:08.545 INFO:teuthology.orchestra.run.smithi107.stdout:1006 punch from 0xfa5600 to 0xfb2400, (0xce00 bytes) 2024-09-11T09:29:08.551 INFO:teuthology.orchestra.run.smithi107.stdout:1008 punch from 0xa2d600 to 0xa39e00, (0xc800 bytes) 2024-09-11T09:29:08.569 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:08.569 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:08.581 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:08.594 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:08.594+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:08.636 INFO:teuthology.orchestra.run.smithi107.stdout:1011 write 0x9c8d400 thru 0x9c9887b (0xb47c bytes) 2024-09-11T09:29:08.640 INFO:teuthology.orchestra.run.smithi107.stdout:1012 write 0x564d600 thru 0x5654e75 (0x7876 bytes) 2024-09-11T09:29:08.640 INFO:teuthology.orchestra.run.smithi107.stdout:1013 read 0x2582400 thru 0x2587ec8 (0x5ac9 bytes) 2024-09-11T09:29:08.641 INFO:teuthology.orchestra.run.smithi107.stdout:1014 punch from 0x83bc000 to 0x83ca000, (0xe000 bytes) 2024-09-11T09:29:08.672 INFO:teuthology.orchestra.run.smithi107.stdout:1016 write 0xe1f1400 thru 0xe1f7369 (0x5f6a bytes) 2024-09-11T09:29:08.673 INFO:teuthology.orchestra.run.smithi107.stdout:1018 write 0x37e6800 thru 0x37f2c55 (0xc456 bytes) 2024-09-11T09:29:08.673 INFO:teuthology.orchestra.run.smithi107.stdout:1021 read 0x6f29200 thru 0x6f354e2 (0xc2e3 bytes) 2024-09-11T09:29:08.673 INFO:teuthology.orchestra.run.smithi107.stdout:1023 write 0x60a2c00 thru 0x60abf90 (0x9391 bytes) 2024-09-11T09:29:08.673 INFO:teuthology.orchestra.run.smithi107.stdout:1024 read 0xcfcd800 thru 0xcfd8506 (0xad07 bytes) 2024-09-11T09:29:08.673 INFO:teuthology.orchestra.run.smithi107.stdout:1026 punch from 0x5167600 to 0x5167800, (0x200 bytes) 2024-09-11T09:29:08.675 INFO:teuthology.orchestra.run.smithi107.stdout:1028 read 0xab1a800 thru 0xab2963a (0xee3b bytes) 2024-09-11T09:29:08.675 INFO:teuthology.orchestra.run.smithi107.stdout:1029 write 0x31a4200 thru 0x31ad233 (0x9034 bytes) 2024-09-11T09:29:08.675 INFO:teuthology.orchestra.run.smithi107.stdout:1032 read 0x9ae8600 thru 0x9aef672 (0x7073 bytes) 2024-09-11T09:29:08.675 INFO:teuthology.orchestra.run.smithi107.stdout:1034 write 0x9a1ae00 thru 0x9a1d0af (0x22b0 bytes) 2024-09-11T09:29:08.676 INFO:teuthology.orchestra.run.smithi107.stdout:1036 punch from 0x7c6000 to 0x7c6e00, (0xe00 bytes) 2024-09-11T09:29:08.679 INFO:teuthology.orchestra.run.smithi107.stdout:1038 trunc from 0xe1f7400 to 0xc993200 2024-09-11T09:29:08.695 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:08.695+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:08.698 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:08.698 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:08.709 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:08.735 INFO:teuthology.orchestra.run.smithi107.stdout:1039 punch from 0x89c5e00 to 0x89d0c00, (0xae00 bytes) 2024-09-11T09:29:08.742 INFO:teuthology.orchestra.run.smithi107.stdout:1043 read 0x184a200 thru 0x1854c5d (0xaa5e bytes) 2024-09-11T09:29:08.742 INFO:teuthology.orchestra.run.smithi107.stdout:1047 write 0xac99200 thru 0xaca3833 (0xa634 bytes) 2024-09-11T09:29:08.742 INFO:teuthology.orchestra.run.smithi107.stdout:1048 read 0x2ed8c00 thru 0x2ee84a6 (0xf8a7 bytes) 2024-09-11T09:29:08.742 INFO:teuthology.orchestra.run.smithi107.stdout:1050 write 0x72f6000 thru 0x730491b (0xe91c bytes) 2024-09-11T09:29:08.744 INFO:teuthology.orchestra.run.smithi107.stdout:1051 trunc from 0xc993200 to 0xad65000 2024-09-11T09:29:08.768 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-11T09:29:08.779 INFO:teuthology.orchestra.run.smithi107.stdout:1053 clone 11 order 22 su 262144 sc 8 2024-09-11T09:29:08.796 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:08.795+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:08.830 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:08.830 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:08.839 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:08.896 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:08.897+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:08.946 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:08.946 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:08.955 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:08.998 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:08.999+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:09.062 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.063 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:09.116 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:09.115+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.175 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.183 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:09.199 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:09.199+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:09.300 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:09.300+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:09.327 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.327 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:09.400 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:09.401+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:09.438 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.438 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:09.501 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:09.500+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:09.550 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.550 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.558 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:09.601 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:09.602+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:09.700 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.700 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.703 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:09.703+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:09.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:09.787 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:29:09.803 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:09.803+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:09.816 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.816 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:09.825 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:09.890 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone10 from 0xad65000 (overlap 0x138a600) to 0x1196200 2024-09-11T09:29:09.905 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:09.905+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:09.996 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:09.997 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:10.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:10.006 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:10.005+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:10.106 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:10.106+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:10.111 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:10.111 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:10.119 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:10.207 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:10.208+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:10.227 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:10.227 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:10.236 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:10.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:10.324 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:10.325+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:10.349 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:10.350 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:10.359 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:10.404 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-11T09:29:10.407 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:10.407 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:10.408 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:10.409+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:10.434 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:10.435 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:10.466 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:10.488 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:10.489 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:10.509 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:10.508+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:10.509 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:10.532 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:10.541 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:10.541 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:10.542 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:10.542 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:10.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:10.610 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:10.609+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:10.711 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:10.711+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.0439062900000007, 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1715 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8386826259999998, 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022344029000000001, 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1715 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.96e-07 2024-09-11T09:29:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876407+0000", 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.871e-06 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876414+0000", 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4679999999999997e-06 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876419+0000", 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1939999999999997e-06 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898563+0000", 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022144646 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898643+0000", 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9900000000000004e-05 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898749+0000", 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105554 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1716 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8386623650000002, 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022552065999999999, 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1716 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876426+0000", 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.77e-07 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876432+0000", 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0079999999999999e-06 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876438+0000", 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4999999999999996e-06 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876443+0000", 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.259e-06 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898781+0000", 2024-09-11T09:29:10.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022337308 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898875+0000", 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4388000000000005e-05 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898977+0000", 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010222599999999999 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1717 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8386374600000002, 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022753961, 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1717 2024-09-11T09:29:10.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876451+0000", 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876452+0000", 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.905e-06 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876459+0000", 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4980000000000003e-06 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876464+0000", 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1909999999999999e-06 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899008+0000", 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022544281999999999 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899093+0000", 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4505000000000005e-05 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899204+0000", 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111049 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1718 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8386167960000002, 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02296196, 2024-09-11T09:29:10.735 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1718 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876473+0000", 2024-09-11T09:29:10.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9259999999999999e-06 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876480+0000", 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876485+0000", 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2159999999999997e-06 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899235+0000", 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022750631 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899321+0000", 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5958999999999995e-05 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899433+0000", 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011126899999999999 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.739 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1719 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 262144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.838576379, 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023149355, 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.740 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1719 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876512+0000", 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.44e-07 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876514+0000", 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876522+0000", 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5000000000000002e-06 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876527+0000", 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.587e-06 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899467+0000", 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022939975000000001 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899552+0000", 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5131999999999997e-05 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899660+0000", 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108137 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1720 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8385527889999995, 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023387654000000001, 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1720 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.63e-07 2024-09-11T09:29:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876537+0000", 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1040000000000001e-06 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876544+0000", 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7800000000000003e-06 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876549+0000", 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2549999999999997e-06 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899706+0000", 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023156708000000002 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899801+0000", 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4826000000000003e-05 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899922+0000", 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121518 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1760 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 12288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8385318970000002, 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023618561999999999, 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1760 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876558+0000", 2024-09-11T09:29:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1670000000000002e-06 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876566+0000", 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3810000000000001e-06 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876571+0000", 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3029999999999999e-06 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899963+0000", 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023392445000000001 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900061+0000", 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7495000000000001e-05 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900174+0000", 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113391 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1761 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8385100970000003, 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023824222999999999, 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1761 2024-09-11T09:29:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876578+0000", 2024-09-11T09:29:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876580+0000", 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9539999999999998e-06 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876586+0000", 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876591+0000", 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900220+0000", 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023628546 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900317+0000", 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7168000000000002e-05 2024-09-11T09:29:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900402+0000", 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4701000000000001e-05 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1762 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 57344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8384898419999995, 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024033523000000001, 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1762 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876600+0000", 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.052e-06 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876607+0000", 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876612+0000", 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9259999999999999e-06 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900448+0000", 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023836570000000001 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900534+0000", 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5314000000000003e-05 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900631+0000", 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7676000000000005e-05 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1763 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 79872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:10.762 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8384694159999997, 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024166084000000001, 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1763 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:10.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876619+0000", 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6400000000000003e-07 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876621+0000", 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0949999999999998e-06 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876629+0000", 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8380000000000008e-06 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876635+0000", 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3199999999999999e-06 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900675+0000", 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024040611 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900724+0000", 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8097999999999999e-05 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900784+0000", 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0658000000000001e-05 2024-09-11T09:29:10.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1764 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.838446179, 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024356692999999999, 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1764 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876642+0000", 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876644+0000", 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.041e-06 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876651+0000", 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6309999999999997e-06 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876656+0000", 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999999e-06 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900814+0000", 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024158129 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900906+0000", 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2618e-05 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900998+0000", 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1624000000000002e-05 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1765 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:10.771 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8384254489999998, 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024574166000000001, 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1765 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:10.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.39e-07 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876664+0000", 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9520000000000001e-06 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876671+0000", 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3670000000000002e-06 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876676+0000", 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:10.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901029+0000", 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024353631000000001 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901109+0000", 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9951999999999996e-05 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901236+0000", 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012697000000000001 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8384056600000003, 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.776 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.780 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8383854280000005, 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:10.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:10.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.784 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8383593019999998, 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:10.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8383370169999997, 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.789 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8383135770000001, 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:10.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.8382937300000002, 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:10.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.802 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.838261353, 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:10.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:10.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:10.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:10.807 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:10.808 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:10.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:10.810 INFO:teuthology.orchestra.run.smithi107.stdout:1055 trunc from 0xad65000 to 0x3b43e00 2024-09-11T09:29:10.810 INFO:teuthology.orchestra.run.smithi107.stdout:1057 trunc from 0x3b43e00 to 0xa157200 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1059 read 0x6498e00 thru 0x649fe73 (0x7074 bytes) 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1060 write 0x2276800 thru 0x2283f1c (0xd71d bytes) 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1061 read 0x913f400 thru 0x914cf84 (0xdb85 bytes) 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1065 trunc from 0xa157200 to 0xe62b000 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1066 write 0x31cd600 thru 0x31d2422 (0x4e23 bytes) 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1068 read 0xd6ce00 thru 0xd7be55 (0xf056 bytes) 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi107.stdout:1069 punch from 0x7edda00 to 0x7ee0000, (0x2600 bytes) 2024-09-11T09:29:10.811 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:29:10.813 INFO:teuthology.orchestra.run.smithi107.stdout:1071 write 0x56a5c00 thru 0x56ab04d (0x544e bytes) 2024-09-11T09:29:10.813 INFO:teuthology.orchestra.run.smithi107.stdout:1072 read 0x2163000 thru 0x216bf1a (0x8f1b bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1076 write 0x43b1200 thru 0x43bf3c4 (0xe1c5 bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1077 read 0xbc0b200 thru 0xbc0c08f (0xe90 bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1079 punch from 0x5902400 to 0x590da00, (0xb600 bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1081 write 0x2349e00 thru 0x234da5b (0x3c5c bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1082 write 0xcdab000 thru 0xcdb10ee (0x60ef bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1089 read 0xa3c4600 thru 0xa3c582e (0x122f bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1090 write 0x1735600 thru 0x173cb4e (0x754f bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1093 read 0x9fad400 thru 0x9fafc6d (0x286e bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1094 read 0xa978600 thru 0xa97f171 (0x6b72 bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1096 write 0x7910c00 thru 0x791b19a (0xa59b bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1098 write 0xb424000 thru 0xb433bb0 (0xfbb1 bytes) 2024-09-11T09:29:10.814 INFO:teuthology.orchestra.run.smithi107.stdout:1099 read 0x3174000 thru 0x3177e26 (0x3e27 bytes) 2024-09-11T09:29:10.815 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:10.812+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:10.815 INFO:teuthology.orchestra.run.smithi107.stdout:1101 write 0x6f79a00 thru 0x6f89670 (0xfc71 bytes) 2024-09-11T09:29:10.815 INFO:teuthology.orchestra.run.smithi107.stdout:1102 read 0xdf33800 thru 0xdf3f850 (0xc051 bytes) 2024-09-11T09:29:10.815 INFO:teuthology.orchestra.run.smithi107.stdout:1105 punch from 0x1534c00 to 0x153aa00, (0x5e00 bytes) 2024-09-11T09:29:10.836 INFO:teuthology.orchestra.run.smithi107.stdout:1106 trunc from 0xe62b000 to 0x590800 2024-09-11T09:29:10.913 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:10.914+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:10.993 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:10.994 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:10.994 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:10.994 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:11.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:11.043 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:11.044+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:11.056 INFO:teuthology.orchestra.run.smithi107.stdout:1108 write 0x56fd600 thru 0x570a76b (0xd16c bytes) 2024-09-11T09:29:11.073 INFO:teuthology.orchestra.run.smithi107.stdout:1109 trunc from 0x570a800 to 0xeb09400 2024-09-11T09:29:11.088 INFO:teuthology.orchestra.run.smithi107.stdout:1110 trunc from 0xeb09400 to 0xbb82800 2024-09-11T09:29:11.100 INFO:teuthology.orchestra.run.smithi107.stdout:1111 read 0x2078600 thru 0x2085f1a (0xd91b bytes) 2024-09-11T09:29:11.114 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:11.115+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:11.115 INFO:teuthology.orchestra.run.smithi107.stdout:1112 write 0xc4c2800 thru 0xc4c58da (0x30db bytes) 2024-09-11T09:29:11.121 INFO:teuthology.orchestra.run.smithi107.stdout:1114 punch from 0x5b1ea00 to 0x5b25200, (0x6800 bytes) 2024-09-11T09:29:11.126 INFO:teuthology.orchestra.run.smithi107.stdout:1116 trunc from 0xc4c5a00 to 0x8181e00 2024-09-11T09:29:11.152 INFO:teuthology.orchestra.run.smithi107.stdout:1117 write 0x98fce00 thru 0x9902c90 (0x5e91 bytes) 2024-09-11T09:29:11.153 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:11.154 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:11.154 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:11.154 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:11.154 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:11.160 INFO:teuthology.orchestra.run.smithi107.stdout:1118 trunc from 0x9902e00 to 0x7e9d400 2024-09-11T09:29:11.163 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:11.181 INFO:teuthology.orchestra.run.smithi107.stdout:1120 write 0x412c400 thru 0x41394fc (0xd0fd bytes) 2024-09-11T09:29:11.181 INFO:teuthology.orchestra.run.smithi107.stdout:1123 read 0xe13c00 thru 0xe1f5c2 (0xb9c3 bytes) 2024-09-11T09:29:11.182 INFO:teuthology.orchestra.run.smithi107.stdout:1125 read 0x549a800 thru 0x54a1bf2 (0x73f3 bytes) 2024-09-11T09:29:11.199 INFO:teuthology.orchestra.run.smithi107.stdout:1131 write 0x97fb400 thru 0x98071e3 (0xbde4 bytes) 2024-09-11T09:29:11.200 INFO:teuthology.orchestra.run.smithi107.stdout:1132 read 0x396ee00 thru 0x3978167 (0x9368 bytes) 2024-09-11T09:29:11.215 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:11.214+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:11.221 INFO:teuthology.orchestra.run.smithi107.stdout:1134 write 0xcc36600 thru 0xcc3a076 (0x3a77 bytes) 2024-09-11T09:29:11.221 INFO:teuthology.orchestra.run.smithi107.stdout:1135 read 0x7128600 thru 0x71360a9 (0xdaaa bytes) 2024-09-11T09:29:11.221 INFO:teuthology.orchestra.run.smithi107.stdout:1137 write 0xa737400 thru 0xa73ee0f (0x7a10 bytes) 2024-09-11T09:29:11.221 INFO:teuthology.orchestra.run.smithi107.stdout:1138 write 0x13f3e00 thru 0x13f5254 (0x1455 bytes) 2024-09-11T09:29:11.222 INFO:teuthology.orchestra.run.smithi107.stdout:1139 read 0xcaf3000 thru 0xcaf87f4 (0x57f5 bytes) 2024-09-11T09:29:11.222 INFO:teuthology.orchestra.run.smithi107.stdout:1141 read 0x36f3e00 thru 0x36fb10c (0x730d bytes) 2024-09-11T09:29:11.224 INFO:teuthology.orchestra.run.smithi107.stdout:1143 trunc from 0xcc3a200 to 0xaceb400 2024-09-11T09:29:11.248 INFO:teuthology.orchestra.run.smithi107.stdout:1147 write 0xd16200 thru 0xd20bc6 (0xa9c7 bytes) 2024-09-11T09:29:11.249 INFO:teuthology.orchestra.run.smithi107.stdout:1149 write 0x8e65000 thru 0x8e6eca5 (0x9ca6 bytes) 2024-09-11T09:29:11.249 INFO:teuthology.orchestra.run.smithi107.stdout:1150 write 0x5e96c00 thru 0x5ea1f8f (0xb390 bytes) 2024-09-11T09:29:11.254 INFO:teuthology.orchestra.run.smithi107.stdout:1155 trunc from 0xaceb400 to 0x6a52200 2024-09-11T09:29:11.276 INFO:teuthology.orchestra.run.smithi107.stdout:1156 read 0x662d200 thru 0x6639cc1 (0xcac2 bytes) 2024-09-11T09:29:11.276 INFO:teuthology.orchestra.run.smithi107.stdout:1157 read 0x179aa00 thru 0x179ec89 (0x428a bytes) 2024-09-11T09:29:11.276 INFO:teuthology.orchestra.run.smithi107.stdout:1158 punch from 0x2a5d600 to 0x2a66000, (0x8a00 bytes) 2024-09-11T09:29:11.285 INFO:teuthology.orchestra.run.smithi107.stdout:1159 trunc from 0x6a52200 to 0xdf56a00 2024-09-11T09:29:11.294 INFO:teuthology.orchestra.run.smithi107.stdout:1160 read 0x1442600 thru 0x144b79f (0x91a0 bytes) 2024-09-11T09:29:11.302 INFO:teuthology.orchestra.run.smithi107.stdout:1163 write 0xb51fc00 thru 0xb524e4b (0x524c bytes) 2024-09-11T09:29:11.302 INFO:teuthology.orchestra.run.smithi107.stdout:1166 read 0xcc28a00 thru 0xcc34a3f (0xc040 bytes) 2024-09-11T09:29:11.302 INFO:teuthology.orchestra.run.smithi107.stdout:1168 write 0x3479c00 thru 0x347b39b (0x179c bytes) 2024-09-11T09:29:11.302 INFO:teuthology.orchestra.run.smithi107.stdout:1172 read 0x1a38000 thru 0x1a3f84c (0x784d bytes) 2024-09-11T09:29:11.303 INFO:teuthology.orchestra.run.smithi107.stdout:1173 write 0x2dc7600 thru 0x2dca2a1 (0x2ca2 bytes) 2024-09-11T09:29:11.313 INFO:teuthology.orchestra.run.smithi107.stdout:1179 trunc from 0xdf56a00 to 0x5a28c00 2024-09-11T09:29:11.316 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:11.315+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:11.321 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40975922199999998, 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40952864999999999, 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40943144599999998, 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.335 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40919668199999998, 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:11.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40900066600000001, 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:11.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:11.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.408732229, 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.345 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:11.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.349 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40863047899999999, 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.350 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.408236822, 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:11.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:11.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:11.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:11.359 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40811057699999997, 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:11.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.364 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40790119499999999, 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.365 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.369 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40779024600000002, 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:11.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40753430800000001, 2024-09-11T09:29:11.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:11.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.407381779, 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:11.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:11.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.407189415, 2024-09-11T09:29:11.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:11.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:11.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.406858997, 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.388 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40673744499999998, 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:11.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40658266700000001, 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:11.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:11.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40652874700000002, 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.402 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.406 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40632817799999998, 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:11.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:11.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:11.411 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.40597263900000002, 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:11.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:11.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.416 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:11.418 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:11.417+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:11.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1181 read 0x1756600 thru 0x175a6ad (0x40ae bytes) 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1183 write 0x710d600 thru 0x7111a7a (0x447b bytes) 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1184 punch from 0xc12c00 to 0xc1f200, (0xc600 bytes) 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1186 trunc from 0x7111c00 to 0x22ef600 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1187 punch from 0x1931e00 to 0x1937c00, (0x5e00 bytes) 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1188 write 0x51f2e00 thru 0x51f715e (0x435f bytes) 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1189 trunc from 0x51f7200 to 0x16e7000 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1190 punch from 0xb3c000 to 0xb42a00, (0x6a00 bytes) 2024-09-11T09:29:11.420 INFO:teuthology.orchestra.run.smithi107.stdout:1191 read 0x1405a00 thru 0x140fa25 (0xa026 bytes) 2024-09-11T09:29:11.421 INFO:teuthology.orchestra.run.smithi107.stdout:1192 write 0x27d5800 thru 0x27e39c2 (0xe1c3 bytes) 2024-09-11T09:29:11.422 INFO:teuthology.orchestra.run.smithi107.stdout:1193 write 0x1bd9400 thru 0x1be79f2 (0xe5f3 bytes) 2024-09-11T09:29:11.423 INFO:teuthology.orchestra.run.smithi107.stdout:1195 punch from 0x355000 to 0x35f200, (0xa200 bytes) 2024-09-11T09:29:11.423 INFO:teuthology.orchestra.run.smithi107.stdout:1196 punch from 0x1a98600 to 0x1aa2800, (0xa200 bytes) 2024-09-11T09:29:11.423 INFO:teuthology.orchestra.run.smithi107.stdout:1200 write 0x189a000 thru 0x18a5c40 (0xbc41 bytes) 2024-09-11T09:29:11.425 INFO:teuthology.orchestra.run.smithi107.stdout:1201 read 0x1648a00 thru 0x164aec4 (0x24c5 bytes) 2024-09-11T09:29:11.425 INFO:teuthology.orchestra.run.smithi107.stdout:1202 punch from 0xef6000 to 0xefe200, (0x8200 bytes) 2024-09-11T09:29:11.426 INFO:teuthology.orchestra.run.smithi107.stdout:1205 read 0x1e67400 thru 0x1e681be (0xdbf bytes) 2024-09-11T09:29:11.426 INFO:teuthology.orchestra.run.smithi107.stdout:1206 read 0x96fa00 thru 0x97a952 (0xaf53 bytes) 2024-09-11T09:29:11.427 INFO:teuthology.orchestra.run.smithi107.stdout:1207 punch from 0x7b7c00 to 0x7b8000, (0x400 bytes) 2024-09-11T09:29:11.427 INFO:teuthology.orchestra.run.smithi107.stdout:1208 read 0x26b5c00 thru 0x26c1375 (0xb776 bytes) 2024-09-11T09:29:11.427 INFO:teuthology.orchestra.run.smithi107.stdout:1210 read 0xe000 thru 0x135fe (0x55ff bytes) 2024-09-11T09:29:11.438 INFO:teuthology.orchestra.run.smithi107.stdout:1214 write 0x3fe7200 thru 0x3fec660 (0x5461 bytes) 2024-09-11T09:29:11.440 INFO:teuthology.orchestra.run.smithi107.stdout:1215 read 0x1457200 thru 0x1466c98 (0xfa99 bytes) 2024-09-11T09:29:11.459 INFO:teuthology.orchestra.run.smithi107.stdout:1216 write 0x6cdce00 thru 0x6cdfe40 (0x3041 bytes) 2024-09-11T09:29:11.463 INFO:teuthology.orchestra.run.smithi107.stdout:1218 trunc from 0x6ce0000 to 0x300f200 2024-09-11T09:29:11.488 INFO:teuthology.orchestra.run.smithi107.stdout:1221 read 0xfea200 thru 0xff20dc (0x7edd bytes) 2024-09-11T09:29:11.488 INFO:teuthology.orchestra.run.smithi107.stdout:1222 write 0x2228600 thru 0x222994a (0x134b bytes) 2024-09-11T09:29:11.488 INFO:teuthology.orchestra.run.smithi107.stdout:1224 read 0xfa3400 thru 0xfaa2f0 (0x6ef1 bytes) 2024-09-11T09:29:11.489 INFO:teuthology.orchestra.run.smithi107.stdout:1225 read 0xe47200 thru 0xe56195 (0xef96 bytes) 2024-09-11T09:29:11.489 INFO:teuthology.orchestra.run.smithi107.stdout:1226 read 0x27eca00 thru 0x27f462b (0x7c2c bytes) 2024-09-11T09:29:11.489 INFO:teuthology.orchestra.run.smithi107.stdout:1227 read 0x1db3e00 thru 0x1dbd72f (0x9930 bytes) 2024-09-11T09:29:11.512 INFO:teuthology.orchestra.run.smithi107.stdout:1228 write 0xae85c00 thru 0xae95a14 (0xfe15 bytes) 2024-09-11T09:29:11.513 INFO:teuthology.orchestra.run.smithi107.stdout:1229 read 0x3ca5400 thru 0x3caec8d (0x988e bytes) 2024-09-11T09:29:11.518 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:11.517+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:11.523 INFO:teuthology.orchestra.run.smithi107.stdout:1233 trunc from 0xae95c00 to 0x24cee00 2024-09-11T09:29:11.537 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:11.537 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:11.537 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:11.537 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:11.546 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:11.547 INFO:teuthology.orchestra.run.smithi107.stdout:1237 read 0x1feba00 thru 0x1ff721f (0xb820 bytes) 2024-09-11T09:29:11.547 INFO:teuthology.orchestra.run.smithi107.stdout:1239 write 0x2070c00 thru 0x2078ff6 (0x83f7 bytes) 2024-09-11T09:29:11.547 INFO:teuthology.orchestra.run.smithi107.stdout:1241 read 0x1ced800 thru 0x1cf130b (0x3b0c bytes) 2024-09-11T09:29:11.547 INFO:teuthology.orchestra.run.smithi107.stdout:1242 punch from 0x1a8c800 to 0x1a92a00, (0x6200 bytes) 2024-09-11T09:29:11.554 INFO:teuthology.orchestra.run.smithi107.stdout:1245 trunc from 0x24cee00 to 0x8918e00 2024-09-11T09:29:11.582 INFO:teuthology.orchestra.run.smithi107.stdout:1246 write 0xc74a000 thru 0xc74a9c2 (0x9c3 bytes) 2024-09-11T09:29:11.583 INFO:teuthology.orchestra.run.smithi107.stdout:1248 clone 12 order 23 su 262144 sc 5 2024-09-11T09:29:11.619 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:11.619+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:11.655 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:11.655 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:11.655 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:11.655 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:11.655 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:11.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:11.720 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:11.721+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:11.771 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.312516940000002, 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:11.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.776 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.050997251999998, 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:11.777 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:11.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:11.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.049751284999999, 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:11.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:11.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.788 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:11.788 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:11.788 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:11.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:11.830 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:11.831+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:11.900 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:11.900 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:11.900 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:11.905 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone11 from 0xc74aa00 (overlap 0x590800) to 0x3e5200 2024-09-11T09:29:11.909 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:11.953 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:11.952+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:12.016 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:12.016 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:12.016 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:12.016 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:12.016 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:12.024 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:12.024+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:12.025 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:12.124 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:12.124+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:12.174 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.737197336999998, 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:12.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:12.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:12.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5081 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2129221430000001, 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043672824999999998, 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5081 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961281+0000", 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961285+0000", 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3560000000000001e-06 2024-09-11T09:29:12.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961287+0000", 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5409999999999999e-06 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004781+0000", 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043493472999999998 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004856+0000", 2024-09-11T09:29:12.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5741000000000005e-05 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004947+0000", 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1222000000000002e-05 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5083 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.212906569, 2024-09-11T09:29:12.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043785219, 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5083 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961291+0000", 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961292+0000", 2024-09-11T09:29:12.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961295+0000", 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0149999999999999e-06 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961297+0000", 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004977+0000", 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043679791000000003 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005021+0000", 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4295000000000002e-05 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005075+0000", 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999998e-05 2024-09-11T09:29:12.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5088 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 905216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2128970219999999, 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043963918999999997, 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5088 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961301+0000", 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.13e-06 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961305+0000", 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4529999999999998e-06 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961309+0000", 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005096+0000", 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043787848999999997 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005175+0000", 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8487999999999998e-05 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005264+0000", 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8786e-05 2024-09-11T09:29:12.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5097 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1605632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2128601750000001, 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044049360000000003, 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5097 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961339+0000", 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1440000000000001e-06 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961345+0000", 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7470000000000002e-06 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961350+0000", 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209999999999996e-06 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005291+0000", 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043940405000000002 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005332+0000", 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1542999999999998e-05 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005386+0000", 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999998e-05 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5100 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:12.200 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2128403240000001, 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044229475999999997, 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5100 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:12.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961357+0000", 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961359+0000", 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.052e-06 2024-09-11T09:29:12.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961365+0000", 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961369+0000", 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6689999999999997e-06 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005422+0000", 2024-09-11T09:29:12.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044052701 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005499+0000", 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6664999999999999e-05 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005586+0000", 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7230000000000003e-05 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5105 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2128217430000001, 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044370891000000003, 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5105 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.205 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:12.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961377+0000", 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9120000000000001e-06 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961383+0000", 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961388+0000", 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005629+0000", 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044240743999999999 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005687+0000", 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7951000000000001e-05 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005746+0000", 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9444999999999998e-05 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5108 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1607680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.212803386, 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581704, 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5108 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961394+0000", 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961396+0000", 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961401+0000", 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8119999999999996e-06 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961406+0000", 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4920000000000004e-06 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005789+0000", 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383498 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005882+0000", 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2541999999999996e-05 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005975+0000", 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3110999999999997e-05 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5111 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2127853420000001, 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044694581999999997, 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5111 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961412+0000", 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:12.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961414+0000", 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.996e-06 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961420+0000", 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8000000000000004e-06 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961425+0000", 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5149999999999997e-06 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006007+0000", 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581786999999998 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006051+0000", 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3998999999999997e-05 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006106+0000", 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5145999999999997e-05 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5116 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1632256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.212765938, 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044886862999999999, 2024-09-11T09:29:12.218 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5116 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961433+0000", 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961439+0000", 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961444+0000", 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809999999999998e-06 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006144+0000", 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044699782 2024-09-11T09:29:12.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006221+0000", 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7270999999999999e-05 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006318+0000", 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6914000000000001e-05 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5121 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1611776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2127475169999999, 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045003543, 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5121 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961450+0000", 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:29:12.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961452+0000", 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0030000000000001e-06 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961459+0000", 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961464+0000", 2024-09-11T09:29:12.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006348+0000", 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044884563000000002 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006396+0000", 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7817e-05 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006453+0000", 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6762000000000003e-05 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5124 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2127270480000001, 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045118734000000001, 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5124 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961472+0000", 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929999999999999e-06 2024-09-11T09:29:12.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961478+0000", 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.592e-06 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961482+0000", 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006470+0000", 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044987680000000002 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006529+0000", 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9116000000000001e-05 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006589+0000", 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9463000000000003e-05 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5129 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1624064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2127086789999999, 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045244390000000002, 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5129 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000002e-07 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961490+0000", 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961496+0000", 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2024-09-11T09:29:12.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961500+0000", 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3819999999999997e-06 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006606+0000", 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045106159 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006669+0000", 2024-09-11T09:29:12.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2188999999999997e-05 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006733+0000", 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3848000000000003e-05 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5134 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2126737080000001, 2024-09-11T09:29:12.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045344917999999998, 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5134 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:12.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961524+0000", 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961531+0000", 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961541+0000", 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0059e-05 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961546+0000", 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006750+0000", 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045204167000000003 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006808+0000", 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8229000000000001e-05 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006868+0000", 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9953e-05 2024-09-11T09:29:12.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5135 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1622016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.212644007, 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045554071000000002, 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5135 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7100000000000002e-07 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961555+0000", 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1679999999999998e-06 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961561+0000", 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961566+0000", 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4170000000000002e-06 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006908+0000", 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045342423999999999 2024-09-11T09:29:12.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007004+0000", 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5307000000000001e-05 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007107+0000", 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103253 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5138 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1634304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2126242380000001, 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045679006000000001, 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5138 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:12.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961805+0000", 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023282700000000001 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961816+0000", 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961828+0000", 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2318e-05 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961835+0000", 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007139+0000", 2024-09-11T09:29:12.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045303738000000003 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007188+0000", 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9014999999999997e-05 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007252+0000", 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3921e-05 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5143 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1613824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2123508540000001, 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045634479999999998, 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5143 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:12.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961994+0000", 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147949 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961997+0000", 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5580000000000001e-06 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962007+0000", 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6579999999999997e-06 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962014+0000", 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007290+0000", 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045276416 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007380+0000", 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9686999999999997e-05 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007480+0000", 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100762 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5148 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1626112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2121729640000001, 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045674845999999998, 2024-09-11T09:29:12.256 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5148 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962412+0000", 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038829000000000001 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962417+0000", 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962427+0000", 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962434+0000", 2024-09-11T09:29:12.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7379999999999997e-06 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007514+0000", 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045080264000000002 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007600+0000", 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5970000000000005e-05 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007699+0000", 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8706e-05 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5153 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1615872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.211751553, 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045448919999999997, 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5153 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962446+0000", 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962448+0000", 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962455+0000", 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962458+0000", 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007712+0000", 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045253466999999999 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007784+0000", 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1939000000000004e-05 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007894+0000", 2024-09-11T09:29:12.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5156 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.2117324949999999, 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045635835999999999, 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:12.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5156 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:12.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962886+0000", 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042155799999999999 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962890+0000", 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962899+0000", 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0440000000000001e-06 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962906+0000", 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1169999999999998e-06 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007917+0000", 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045011412000000001 2024-09-11T09:29:12.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007994+0000", 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6650999999999995e-05 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.008100+0000", 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106164 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:12.271 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:12.271 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:12.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:12.272 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:12.272+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:12.372 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:12.372+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:12.408 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-11T09:29:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:12.423 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:12.423 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:12.435 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:12.476 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:12.476+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:12.548 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:12.574 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:12.573+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:12.625 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:12.626 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:12.626 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:12.626 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:12.626 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:12.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:12.677 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:12.677+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:12.685 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:12.685 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:12.702 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:12.702 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:12.716 INFO:teuthology.orchestra.run.smithi107.stdout:1249 trunc from 0xc74aa00 to 0x221ea00 2024-09-11T09:29:12.718 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:12.718 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:12.740 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:12.741 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:12.758 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:12.759 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:12.775 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:12.775 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:12.775 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:12.775+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:12.787 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:12.788 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:12.796 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:12.812 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:29:12.876 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:12.876+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:12.898 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:12.898 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:12.907 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:12.977 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:12.977+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:12.992 INFO:teuthology.orchestra.run.smithi107.stdout:1250 read 0xc06a00 thru 0xc0f451 (0x8a52 bytes) 2024-09-11T09:29:12.992 INFO:teuthology.orchestra.run.smithi107.stdout:1251 read 0x1a26400 thru 0x1a28d83 (0x2984 bytes) 2024-09-11T09:29:12.993 INFO:teuthology.orchestra.run.smithi107.stdout:1252 read 0x3a4400 thru 0x3abc9c (0x789d bytes) 2024-09-11T09:29:13.011 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.011 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.021 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:13.023 INFO:teuthology.orchestra.run.smithi107.stdout:1253 write 0xdfdf400 thru 0xdfe6e47 (0x7a48 bytes) 2024-09-11T09:29:13.023 INFO:teuthology.orchestra.run.smithi107.stdout:1254 write 0x4d14c00 thru 0x4d1a817 (0x5c18 bytes) 2024-09-11T09:29:13.023 INFO:teuthology.orchestra.run.smithi107.stdout:1255 write 0x69efa00 thru 0x69ff741 (0xfd42 bytes) 2024-09-11T09:29:13.023 INFO:teuthology.orchestra.run.smithi107.stdout:1256 read 0x161c800 thru 0x1625b79 (0x937a bytes) 2024-09-11T09:29:13.024 INFO:teuthology.orchestra.run.smithi107.stdout:1260 punch from 0x6a16800 to 0x6a1fc00, (0x9400 bytes) 2024-09-11T09:29:13.025 INFO:teuthology.orchestra.run.smithi107.stdout:1261 write 0xa7a0000 thru 0xa7afe0d (0xfe0e bytes) 2024-09-11T09:29:13.034 INFO:teuthology.orchestra.run.smithi107.stdout:1262 trunc from 0xdfe7000 to 0x61ba400 2024-09-11T09:29:13.059 INFO:teuthology.orchestra.run.smithi107.stdout:1263 write 0x5d77200 thru 0x5d862a7 (0xf0a8 bytes) 2024-09-11T09:29:13.059 INFO:teuthology.orchestra.run.smithi107.stdout:1265 punch from 0x1319600 to 0x131a600, (0x1000 bytes) 2024-09-11T09:29:13.077 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:13.077+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:13.079 INFO:teuthology.orchestra.run.smithi107.stdout:1269 write 0xaf24e00 thru 0xaf2766f (0x2870 bytes) 2024-09-11T09:29:13.080 INFO:teuthology.orchestra.run.smithi107.stdout:1270 trunc from 0xaf27800 to 0xbd58a00 2024-09-11T09:29:13.101 INFO:teuthology.orchestra.run.smithi107.stdout:1272 trunc from 0xbd58a00 to 0x358c400 2024-09-11T09:29:13.116 INFO:teuthology.orchestra.run.smithi107.stdout:1273 read 0x1c2e400 thru 0x1c2e76f (0x370 bytes) 2024-09-11T09:29:13.117 INFO:teuthology.orchestra.run.smithi107.stdout:1274 punch from 0x185600 to 0x18c400, (0x6e00 bytes) 2024-09-11T09:29:13.133 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.133 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.141 INFO:teuthology.orchestra.run.smithi107.stdout:1275 write 0xcf21c00 thru 0xcf23a12 (0x1e13 bytes) 2024-09-11T09:29:13.142 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:13.143 INFO:teuthology.orchestra.run.smithi107.stdout:1276 punch from 0x1025a00 to 0x1026e00, (0x1400 bytes) 2024-09-11T09:29:13.149 INFO:teuthology.orchestra.run.smithi107.stdout:1277 read 0x2253800 thru 0x225d22d (0x9a2e bytes) 2024-09-11T09:29:13.149 INFO:teuthology.orchestra.run.smithi107.stdout:1279 write 0x9ab8e00 thru 0x9ac4e1e (0xc01f bytes) 2024-09-11T09:29:13.152 INFO:teuthology.orchestra.run.smithi107.stdout:1282 trunc from 0xcf23c00 to 0x88c4600 2024-09-11T09:29:13.171 INFO:teuthology.orchestra.run.smithi107.stdout:1284 read 0x5500000 thru 0x550cf2e (0xcf2f bytes) 2024-09-11T09:29:13.172 INFO:teuthology.orchestra.run.smithi107.stdout:truncating to largest ever: 0xee2c800 2024-09-11T09:29:13.180 INFO:teuthology.orchestra.run.smithi107.stdout:1288 trunc from 0x88c4600 to 0xee2c800 2024-09-11T09:29:13.189 INFO:teuthology.orchestra.run.smithi107.stdout:1292 punch from 0x4181c00 to 0x4186800, (0x4c00 bytes) 2024-09-11T09:29:13.190 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:13.191+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:13.191 INFO:teuthology.orchestra.run.smithi107.stdout:1293 read 0xbd1800 thru 0xbdd1f6 (0xb9f7 bytes) 2024-09-11T09:29:13.199 INFO:teuthology.orchestra.run.smithi107.stdout:1295 trunc from 0xee2c800 to 0x8a4a000 2024-09-11T09:29:13.210 INFO:teuthology.orchestra.run.smithi107.stdout:1296 trunc from 0x8a4a000 to 0xa367c00 2024-09-11T09:29:13.221 INFO:teuthology.orchestra.run.smithi107.stdout:1297 write 0x7ecde00 thru 0x7ecf597 (0x1798 bytes) 2024-09-11T09:29:13.221 INFO:teuthology.orchestra.run.smithi107.stdout:1304 punch from 0x40d7600 to 0x40e6400, (0xee00 bytes) 2024-09-11T09:29:13.223 INFO:teuthology.orchestra.run.smithi107.stdout:1306 flatten 2024-09-11T09:29:13.235 INFO:teuthology.orchestra.run.smithi107.stdout:1309 write 0x4ad2e00 thru 0x4adaff5 (0x81f6 bytes) 2024-09-11T09:29:13.236 INFO:teuthology.orchestra.run.smithi107.stdout:1310 read 0x62e0400 thru 0x62e832f (0x7f30 bytes) 2024-09-11T09:29:13.236 INFO:teuthology.orchestra.run.smithi107.stdout:1312 write 0x844ec00 thru 0x8458b63 (0x9f64 bytes) 2024-09-11T09:29:13.237 INFO:teuthology.orchestra.run.smithi107.stdout:1313 punch from 0x1b6c000 to 0x1b76a00, (0xaa00 bytes) 2024-09-11T09:29:13.237 INFO:teuthology.orchestra.run.smithi107.stdout:1314 punch from 0x19de800 to 0x19ec200, (0xda00 bytes) 2024-09-11T09:29:13.237 INFO:teuthology.orchestra.run.smithi107.stdout:1316 punch from 0x7033400 to 0x7041e00, (0xea00 bytes) 2024-09-11T09:29:13.243 INFO:teuthology.orchestra.run.smithi107.stdout:1317 trunc from 0xa367c00 to 0x5c9d800 2024-09-11T09:29:13.258 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.258 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.267 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:13.270 INFO:teuthology.orchestra.run.smithi107.stdout:1320 write 0x9d33800 thru 0x9d3409d (0x89e bytes) 2024-09-11T09:29:13.279 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:13.279+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:13.290 INFO:teuthology.orchestra.run.smithi107.stdout:1325 write 0xb70a400 thru 0xb70af17 (0xb18 bytes) 2024-09-11T09:29:13.297 INFO:teuthology.orchestra.run.smithi107.stdout:1326 trunc from 0xb70b000 to 0xc09c800 2024-09-11T09:29:13.311 INFO:teuthology.orchestra.run.smithi107.stdout:1327 trunc from 0xc09c800 to 0xda10000 2024-09-11T09:29:13.319 INFO:teuthology.orchestra.run.smithi107.stdout:1330 write 0xb1ce800 thru 0xb1d9d16 (0xb517 bytes) 2024-09-11T09:29:13.320 INFO:teuthology.orchestra.run.smithi107.stdout:1333 punch from 0x1175c00 to 0x1179600, (0x3a00 bytes) 2024-09-11T09:29:13.327 INFO:teuthology.orchestra.run.smithi107.stdout:1337 trunc from 0xda10000 to 0x8a0d200 2024-09-11T09:29:13.341 INFO:teuthology.orchestra.run.smithi107.stdout:1339 read 0x5136000 thru 0x5140ede (0xaedf bytes) 2024-09-11T09:29:13.344 INFO:teuthology.orchestra.run.smithi107.stdout:1341 trunc from 0x8a0d200 to 0x6fdec00 2024-09-11T09:29:13.372 INFO:teuthology.orchestra.run.smithi107.stdout:1342 write 0xcf69c00 thru 0xcf711f9 (0x75fa bytes) 2024-09-11T09:29:13.373 INFO:teuthology.orchestra.run.smithi107.stdout:1344 write 0x2568200 thru 0x257490f (0xc710 bytes) 2024-09-11T09:29:13.373 INFO:teuthology.orchestra.run.smithi107.stdout:1345 write 0x72e1200 thru 0x72e3afd (0x28fe bytes) 2024-09-11T09:29:13.379 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:13.380+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:13.389 INFO:teuthology.orchestra.run.smithi107.stdout:1348 trunc from 0xcf71200 to 0xcd200 2024-09-11T09:29:13.410 INFO:teuthology.orchestra.run.smithi107.stdout:1349 read 0x9c600 thru 0xa567f (0x9080 bytes) 2024-09-11T09:29:13.421 INFO:teuthology.orchestra.run.smithi107.stdout:1351 trunc from 0xcd200 to 0x80f5000 2024-09-11T09:29:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.429 INFO:teuthology.orchestra.run.smithi107.stdout:1353 read 0x5893400 thru 0x5898730 (0x5331 bytes) 2024-09-11T09:29:13.431 INFO:teuthology.orchestra.run.smithi107.stdout:1354 write 0x73fd200 thru 0x73ff834 (0x2635 bytes) 2024-09-11T09:29:13.431 INFO:teuthology.orchestra.run.smithi107.stdout:1355 write 0x29f9200 thru 0x29fde1b (0x4c1c bytes) 2024-09-11T09:29:13.431 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:13.437 INFO:teuthology.orchestra.run.smithi107.stdout:1356 trunc from 0x80f5000 to 0x2175a00 2024-09-11T09:29:13.466 INFO:teuthology.orchestra.run.smithi107.stdout:1357 write 0x21d8a00 thru 0x21e0d41 (0x8342 bytes) 2024-09-11T09:29:13.480 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:13.479+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:13.487 INFO:teuthology.orchestra.run.smithi107.stdout:1358 write 0x669b000 thru 0x66a8a3a (0xda3b bytes) 2024-09-11T09:29:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:1359 write 0xcf05000 thru 0xcf11f9b (0xcf9c bytes) 2024-09-11T09:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:1360 punch from 0x96ea400 to 0x96f0600, (0x6200 bytes) 2024-09-11T09:29:13.514 INFO:teuthology.orchestra.run.smithi107.stdout:1361 read 0x2d58c00 thru 0x2d5cbd9 (0x3fda bytes) 2024-09-11T09:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:1362 write 0xa39a200 thru 0xa39b1da (0xfdb bytes) 2024-09-11T09:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:1364 read 0xba80a00 thru 0xba8ce66 (0xc467 bytes) 2024-09-11T09:29:13.517 INFO:teuthology.orchestra.run.smithi107.stdout:1366 read 0x3f43800 thru 0x3f5342d (0xfc2e bytes) 2024-09-11T09:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:1367 write 0x182b000 thru 0x1832ae1 (0x7ae2 bytes) 2024-09-11T09:29:13.518 INFO:teuthology.orchestra.run.smithi107.stdout:1368 write 0x516a000 thru 0x516ca28 (0x2a29 bytes) 2024-09-11T09:29:13.521 INFO:teuthology.orchestra.run.smithi107.stdout:1370 trunc from 0xcf12000 to 0x787a400 2024-09-11T09:29:13.552 INFO:teuthology.orchestra.run.smithi107.stdout:1372 trunc from 0x787a400 to 0x63b5800 2024-09-11T09:29:13.568 INFO:teuthology.orchestra.run.smithi107.stdout:1373 punch from 0x1e45e00 to 0x1e55c00, (0xfe00 bytes) 2024-09-11T09:29:13.581 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:13.581+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:13.584 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.584 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.586 INFO:teuthology.orchestra.run.smithi107.stdout:1375 write 0x96b1800 thru 0x96b94fa (0x7cfb bytes) 2024-09-11T09:29:13.593 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:13.601 INFO:teuthology.orchestra.run.smithi107.stdout:1376 write 0xc99d000 thru 0xc9a1ff5 (0x4ff6 bytes) 2024-09-11T09:29:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:1379 write 0x75200 thru 0x76a16 (0x1817 bytes) 2024-09-11T09:29:13.614 INFO:teuthology.orchestra.run.smithi107.stdout:1381 trunc from 0xc9a2000 to 0x5bfd400 2024-09-11T09:29:13.656 INFO:teuthology.orchestra.run.smithi107.stdout:1382 write 0x9f07400 thru 0x9f08158 (0xd59 bytes) 2024-09-11T09:29:13.656 INFO:teuthology.orchestra.run.smithi107.stdout:1383 punch from 0x26de200 to 0x26e0600, (0x2400 bytes) 2024-09-11T09:29:13.656 INFO:teuthology.orchestra.run.smithi107.stdout:1386 read 0x79be000 thru 0x79ca191 (0xc192 bytes) 2024-09-11T09:29:13.672 INFO:teuthology.orchestra.run.smithi107.stdout:1387 write 0xbc5a600 thru 0xbc68bee (0xe5ef bytes) 2024-09-11T09:29:13.672 INFO:teuthology.orchestra.run.smithi107.stdout:1388 write 0x557ee00 thru 0x55876e4 (0x88e5 bytes) 2024-09-11T09:29:13.673 INFO:teuthology.orchestra.run.smithi107.stdout:1389 write 0x9b9600 thru 0x9c5be9 (0xc5ea bytes) 2024-09-11T09:29:13.673 INFO:teuthology.orchestra.run.smithi107.stdout:1390 read 0x1ecfc00 thru 0x1ed96d4 (0x9ad5 bytes) 2024-09-11T09:29:13.673 INFO:teuthology.orchestra.run.smithi107.stdout:1396 read 0x69e8400 thru 0x69e94f5 (0x10f6 bytes) 2024-09-11T09:29:13.673 INFO:teuthology.orchestra.run.smithi107.stdout:1398 read 0x8734600 thru 0x873dae6 (0x94e7 bytes) 2024-09-11T09:29:13.673 INFO:teuthology.orchestra.run.smithi107.stdout:1400 write 0x3d02200 thru 0x3d08fc3 (0x6dc4 bytes) 2024-09-11T09:29:13.674 INFO:teuthology.orchestra.run.smithi107.stdout:1402 trunc from 0xbc68c00 to 0xaabfa00 2024-09-11T09:29:13.682 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:13.682+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:13.697 INFO:teuthology.orchestra.run.smithi107.stdout:1405 trunc from 0xaabfa00 to 0xd1a3600 2024-09-11T09:29:13.707 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.707 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.711 INFO:teuthology.orchestra.run.smithi107.stdout:1406 punch from 0x24e1200 to 0x24eae00, (0x9c00 bytes) 2024-09-11T09:29:13.712 INFO:teuthology.orchestra.run.smithi107.stdout:1408 read 0x3d11600 thru 0x3d1fd14 (0xe715 bytes) 2024-09-11T09:29:13.712 INFO:teuthology.orchestra.run.smithi107.stdout:1409 write 0x61c4e00 thru 0x61cb65d (0x685e bytes) 2024-09-11T09:29:13.712 INFO:teuthology.orchestra.run.smithi107.stdout:1411 punch from 0x8bc5a00 to 0x8bc7e00, (0x2400 bytes) 2024-09-11T09:29:13.713 INFO:teuthology.orchestra.run.smithi107.stdout:1412 clone 13 order 18 su 65536 sc 12 2024-09-11T09:29:13.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:13.783 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:13.783 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:13.783 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-11T09:29:13.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-11T09:29:13.784 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:13.783+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:13.787 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:29:13.831 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.831 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.841 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:13.923 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:13.924 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:13.923+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:13.925 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:13.926 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:13.926 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-11T09:29:13.985 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:13.986+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:13.990 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:13.990 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:13.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:14.086 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:14.087+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:14.104 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:14.108 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:14.108 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:14.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-11T09:29:14.153 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:14.154 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:14.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:14.203 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:14.204+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:14.286 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:14.286 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:14.286 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:14.286 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:14.286 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:14.287 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:14.288 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:14.288+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:14.289 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:14.289 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:14.290 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-11T09:29:14.314 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:14.314 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:14.322 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-11T09:29:14.323 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout:1414 read 0x7354200 thru 0x735e578 (0xa379 bytes) 2024-09-11T09:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout:1415 read 0xbb36c00 thru 0xbb389ba (0x1dbb bytes) 2024-09-11T09:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout:1416 read 0x205600 thru 0x212fc0 (0xd9c1 bytes) 2024-09-11T09:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout:1417 trunc from 0xd1a3600 to 0xce7f400 2024-09-11T09:29:14.407 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:14.408+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:14.425 INFO:teuthology.orchestra.run.smithi107.stdout:1418 write 0x17aa000 thru 0x17b1581 (0x7582 bytes) 2024-09-11T09:29:14.427 INFO:teuthology.orchestra.run.smithi107.stdout:1420 write 0x153d200 thru 0x154c2e2 (0xf0e3 bytes) 2024-09-11T09:29:14.427 INFO:teuthology.orchestra.run.smithi107.stdout:1423 punch from 0x4948c00 to 0x4958a00, (0xfe00 bytes) 2024-09-11T09:29:14.429 INFO:teuthology.orchestra.run.smithi107.stdout:1424 read 0xa052600 thru 0xa05e514 (0xbf15 bytes) 2024-09-11T09:29:14.429 INFO:teuthology.orchestra.run.smithi107.stdout:1425 write 0x49a9800 thru 0x49ab48f (0x1c90 bytes) 2024-09-11T09:29:14.432 INFO:teuthology.orchestra.run.smithi107.stdout:1426 trunc from 0xce7f400 to 0xae8a200 2024-09-11T09:29:14.458 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:14.458 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:14.459 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:14.462 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:14.462 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:14.462 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-11T09:29:14.477 INFO:teuthology.orchestra.run.smithi107.stdout:1427 trunc from 0xae8a200 to 0x1e9cc00 2024-09-11T09:29:14.490 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:14.490+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:14.505 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:14.505 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:1430 punch from 0x506c00 to 0x50d400, (0x6800 bytes) 2024-09-11T09:29:14.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:14.539 INFO:teuthology.orchestra.run.smithi107.stdout:1437 write 0xb80f400 thru 0xb81d1d6 (0xddd7 bytes) 2024-09-11T09:29:14.544 INFO:teuthology.orchestra.run.smithi107.stdout:1438 read 0x520200 thru 0x52dc26 (0xda27 bytes) 2024-09-11T09:29:14.546 INFO:teuthology.orchestra.run.smithi107.stdout:1441 write 0x2415600 thru 0x241be01 (0x6802 bytes) 2024-09-11T09:29:14.546 INFO:teuthology.orchestra.run.smithi107.stdout:1442 punch from 0x9399000 to 0x93a1400, (0x8400 bytes) 2024-09-11T09:29:14.546 INFO:teuthology.orchestra.run.smithi107.stdout:1444 write 0x8e45200 thru 0x8e4b8d7 (0x66d8 bytes) 2024-09-11T09:29:14.549 INFO:teuthology.orchestra.run.smithi107.stdout:1445 trunc from 0xb81d200 to 0xd912600 2024-09-11T09:29:14.567 INFO:teuthology.orchestra.run.smithi107.stdout:1449 punch from 0x3b2fe00 to 0x3b30c00, (0xe00 bytes) 2024-09-11T09:29:14.569 INFO:teuthology.orchestra.run.smithi107.stdout:1451 read 0x6320800 thru 0x632f7e9 (0xefea bytes) 2024-09-11T09:29:14.569 INFO:teuthology.orchestra.run.smithi107.stdout:1452 read 0x49ca000 thru 0x49d74f3 (0xd4f4 bytes) 2024-09-11T09:29:14.581 INFO:teuthology.orchestra.run.smithi107.stdout:1454 trunc from 0xd912600 to 0x3251600 2024-09-11T09:29:14.591 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:14.591+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:14.609 INFO:teuthology.orchestra.run.smithi107.stdout:1455 punch from 0x1702e00 to 0x170ac00, (0x7e00 bytes) 2024-09-11T09:29:14.609 INFO:teuthology.orchestra.run.smithi107.stdout:1456 read 0x617a00 thru 0x6262f8 (0xe8f9 bytes) 2024-09-11T09:29:14.625 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:14.625 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:14.627 INFO:teuthology.orchestra.run.smithi107.stdout:1457 write 0x358d400 thru 0x358fd6e (0x296f bytes) 2024-09-11T09:29:14.627 INFO:teuthology.orchestra.run.smithi107.stdout:1458 read 0x1d31000 thru 0x1d38148 (0x7149 bytes) 2024-09-11T09:29:14.634 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:14.650 INFO:teuthology.orchestra.run.smithi107.stdout:1459 write 0x796d600 thru 0x796dfe8 (0x9e9 bytes) 2024-09-11T09:29:14.650 INFO:teuthology.orchestra.run.smithi107.stdout:1460 punch from 0x1eef800 to 0x1ef8e00, (0x9600 bytes) 2024-09-11T09:29:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:29:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:29:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:29:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:29:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:29:14.656 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:29:14.657 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:29:14.657 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:29:14.657 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:29:14.657 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:29:14.657 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:29:14.657 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:29:14.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:29:14.659 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:29:14.673 INFO:teuthology.orchestra.run.smithi107.stdout:1461 write 0xb507a00 thru 0xb516242 (0xe843 bytes) 2024-09-11T09:29:14.673 INFO:teuthology.orchestra.run.smithi107.stdout:1467 read 0x5d7b200 thru 0x5d825e3 (0x73e4 bytes) 2024-09-11T09:29:14.692 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:14.692+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1469 write 0xeba3200 thru 0xeba5219 (0x201a bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1471 punch from 0x8450800 to 0x8460600, (0xfe00 bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1472 punch from 0xc6cc600 to 0xc6d4600, (0x8000 bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1474 write 0x27bd600 thru 0x27c2043 (0x4a44 bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1476 punch from 0xd7ede00 to 0xd7f7e00, (0xa000 bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1478 punch from 0x798ca00 to 0x7998a00, (0xc000 bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1482 punch from 0x6f85a00 to 0x6f8a200, (0x4800 bytes) 2024-09-11T09:29:14.696 INFO:teuthology.orchestra.run.smithi107.stdout:1483 write 0x10cb200 thru 0x10d4f52 (0x9d53 bytes) 2024-09-11T09:29:14.697 INFO:teuthology.orchestra.run.smithi107.stdout:1484 punch from 0x27c1200 to 0x27c3e00, (0x2c00 bytes) 2024-09-11T09:29:14.709 INFO:teuthology.orchestra.run.smithi107.stdout:1485 read 0x799d000 thru 0x79abeea (0xeeeb bytes) 2024-09-11T09:29:14.715 INFO:teuthology.orchestra.run.smithi107.stdout:1487 write 0xc39dc00 thru 0xc3a7c1e (0xa01f bytes) 2024-09-11T09:29:14.715 INFO:teuthology.orchestra.run.smithi107.stdout:1488 punch from 0x47eac00 to 0x47eb000, (0x400 bytes) 2024-09-11T09:29:14.715 INFO:teuthology.orchestra.run.smithi107.stdout:1489 read 0xc286a00 thru 0xc28f17d (0x877e bytes) 2024-09-11T09:29:14.715 INFO:teuthology.orchestra.run.smithi107.stdout:1490 write 0x24cf200 thru 0x24dba9a (0xc89b bytes) 2024-09-11T09:29:14.716 INFO:teuthology.orchestra.run.smithi107.stdout:1492 punch from 0x7f54a00 to 0x7f56400, (0x1a00 bytes) 2024-09-11T09:29:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:1493 trunc from 0xeba5400 to 0x4e61000 2024-09-11T09:29:14.754 INFO:teuthology.orchestra.run.smithi107.stdout:1494 read 0x39d3e00 thru 0x39db228 (0x7429 bytes) 2024-09-11T09:29:14.754 INFO:teuthology.orchestra.run.smithi107.stdout:1499 read 0xf7c400 thru 0xf8b51d (0xf11e bytes) 2024-09-11T09:29:14.756 INFO:teuthology.orchestra.run.smithi107.stdout:1500 trunc from 0x4e61000 to 0x6580c00 2024-09-11T09:29:14.764 INFO:teuthology.orchestra.run.smithi107.stdout:1502 write 0x28f5e00 thru 0x2903b7e (0xdd7f bytes) 2024-09-11T09:29:14.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:14.789 INFO:teuthology.orchestra.run.smithi107.stdout:1504 write 0xeb4d400 thru 0xeb5a5e9 (0xd1ea bytes) 2024-09-11T09:29:14.790 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:14.790 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:14.792 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:14.792+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:14.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:14.799 INFO:teuthology.orchestra.run.smithi107.stdout:1505 trunc from 0xeb5a600 to 0x6a8a800 2024-09-11T09:29:14.818 INFO:teuthology.orchestra.run.smithi107.stdout:1510 read 0x2b00000 thru 0x2b0273d (0x273e bytes) 2024-09-11T09:29:14.833 INFO:teuthology.orchestra.run.smithi107.stdout:1511 write 0xb7fea00 thru 0xb80d23e (0xe83f bytes) 2024-09-11T09:29:14.847 INFO:teuthology.orchestra.run.smithi107.stdout:1513 trunc from 0xb80d400 to 0x4ab5a00 2024-09-11T09:29:14.868 INFO:teuthology.orchestra.run.smithi107.stdout:1515 read 0x3c44800 thru 0x3c4d56b (0x8d6c bytes) 2024-09-11T09:29:14.874 INFO:teuthology.orchestra.run.smithi107.stdout:1516 read 0x5fc600 thru 0x60258a (0x5f8b bytes) 2024-09-11T09:29:14.890 INFO:teuthology.orchestra.run.smithi107.stdout:1517 write 0x9974000 thru 0x997b79a (0x779b bytes) 2024-09-11T09:29:14.892 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:14.893+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:14.897 INFO:teuthology.orchestra.run.smithi107.stdout:1518 trunc from 0x997b800 to 0x8abb400 2024-09-11T09:29:14.925 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:14.925 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:14.940 INFO:teuthology.orchestra.run.smithi107.stdout:1519 write 0xb0e5c00 thru 0xb0edcd7 (0x80d8 bytes) 2024-09-11T09:29:14.944 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:14.945 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:14.947 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:14.947 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:14.956 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:14.962 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:14.962 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:14.963 INFO:teuthology.orchestra.run.smithi107.stdout:1520 write 0xe3e5400 thru 0xe3efad4 (0xa6d5 bytes) 2024-09-11T09:29:14.963 INFO:teuthology.orchestra.run.smithi107.stdout:1521 write 0x44ed000 thru 0x44f0cd4 (0x3cd5 bytes) 2024-09-11T09:29:14.971 INFO:teuthology.orchestra.run.smithi107.stdout:1523 trunc from 0xe3efc00 to 0x7371400 2024-09-11T09:29:14.981 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:14.981 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:14.989 INFO:teuthology.orchestra.run.smithi107.stdout:1526 punch from 0x68a000 to 0x692000, (0x8000 bytes) 2024-09-11T09:29:15.000 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:14.998+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:15.001 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:15.001 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:15.001 INFO:teuthology.orchestra.run.smithi107.stdout:1527 punch from 0x4c5c000 to 0x4c61000, (0x5000 bytes) 2024-09-11T09:29:15.002 INFO:teuthology.orchestra.run.smithi107.stdout:1528 write 0x3082200 thru 0x3087b0c (0x590d bytes) 2024-09-11T09:29:15.002 INFO:teuthology.orchestra.run.smithi107.stdout:1529 punch from 0x29e9800 to 0x29f8a00, (0xf200 bytes) 2024-09-11T09:29:15.002 INFO:teuthology.orchestra.run.smithi107.stdout:1533 read 0xba7e00 thru 0xbb2122 (0xa323 bytes) 2024-09-11T09:29:15.002 INFO:teuthology.orchestra.run.smithi107.stdout:1535 read 0xeae800 thru 0xebe3db (0xfbdc bytes) 2024-09-11T09:29:15.017 INFO:teuthology.orchestra.run.smithi107.stdout:1536 write 0x902e200 thru 0x90394ae (0xb2af bytes) 2024-09-11T09:29:15.018 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:15.018 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:15.023 INFO:teuthology.orchestra.run.smithi107.stdout:1537 punch from 0x7d98000 to 0x7d9fa00, (0x7a00 bytes) 2024-09-11T09:29:15.025 INFO:teuthology.orchestra.run.smithi107.stdout:1538 write 0x677e600 thru 0x678d696 (0xf097 bytes) 2024-09-11T09:29:15.025 INFO:teuthology.orchestra.run.smithi107.stdout:1540 read 0x5038a00 thru 0x5040ce8 (0x82e9 bytes) 2024-09-11T09:29:15.026 INFO:teuthology.orchestra.run.smithi107.stdout:1541 trunc from 0x9039600 to 0x9409200 2024-09-11T09:29:15.048 INFO:teuthology.orchestra.run.smithi107.stdout:1546 write 0x78ffa00 thru 0x790cbcc (0xd1cd bytes) 2024-09-11T09:29:15.049 INFO:teuthology.orchestra.run.smithi107.stdout:1547 read 0x86b7600 thru 0x86c1a92 (0xa493 bytes) 2024-09-11T09:29:15.049 INFO:teuthology.orchestra.run.smithi107.stdout:1548 read 0x4596c00 thru 0x45a32a8 (0xc6a9 bytes) 2024-09-11T09:29:15.051 INFO:teuthology.orchestra.run.smithi107.stdout:1549 trunc from 0x9409200 to 0xb63aa00 2024-09-11T09:29:15.066 INFO:teuthology.orchestra.run.smithi107.stdout:1550 punch from 0x7261400 to 0x726d600, (0xc200 bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1552 write 0xe16d000 thru 0xe16fb85 (0x2b86 bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1553 read 0xcd02a00 thru 0xcd0858a (0x5b8b bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1554 write 0x7397e00 thru 0x739936c (0x156d bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1558 write 0xa3c2a00 thru 0xa3c697d (0x3f7e bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1559 write 0x53f3a00 thru 0x53f8c91 (0x5292 bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1560 write 0x2e8e200 thru 0x2e9c153 (0xdf54 bytes) 2024-09-11T09:29:15.092 INFO:teuthology.orchestra.run.smithi107.stdout:1561 punch from 0x238c000 to 0x2395400, (0x9400 bytes) 2024-09-11T09:29:15.093 INFO:teuthology.orchestra.run.smithi107.stdout:1563 punch from 0xb635c00 to 0xb643400, (0xd800 bytes) 2024-09-11T09:29:15.093 INFO:teuthology.orchestra.run.smithi107.stdout:1565 punch from 0x13bc400 to 0x13c2e00, (0x6a00 bytes) 2024-09-11T09:29:15.093 INFO:teuthology.orchestra.run.smithi107.stdout:1566 punch from 0x9956600 to 0x9962c00, (0xc600 bytes) 2024-09-11T09:29:15.094 INFO:teuthology.orchestra.run.smithi107.stdout:1572 trunc from 0xe16fc00 to 0xd27fa00 2024-09-11T09:29:15.094 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:15.094+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:15.114 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:15.114 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:15.114 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:15.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:15.140 INFO:teuthology.orchestra.run.smithi107.stdout:1574 trunc from 0xd27fa00 to 0x4e9c00 2024-09-11T09:29:15.195 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:15.196+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:15.200 INFO:teuthology.orchestra.run.smithi107.stdout:1577 read 0x445a00 thru 0x449796 (0x3d97 bytes) 2024-09-11T09:29:15.235 INFO:teuthology.orchestra.run.smithi107.stdout:1578 write 0xdec5600 thru 0xdec7fc6 (0x29c7 bytes) 2024-09-11T09:29:15.235 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:15.236 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:15.236 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:15.236 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:15.236 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:15.242 INFO:teuthology.orchestra.run.smithi107.stdout:1579 write 0x6d88a00 thru 0x6d8f41c (0x6a1d bytes) 2024-09-11T09:29:15.242 INFO:teuthology.orchestra.run.smithi107.stdout:1581 read 0x30af400 thru 0x30bdda6 (0xe9a7 bytes) 2024-09-11T09:29:15.242 INFO:teuthology.orchestra.run.smithi107.stdout:1584 punch from 0x8a43e00 to 0x8a44000, (0x200 bytes) 2024-09-11T09:29:15.244 INFO:teuthology.orchestra.run.smithi107.stdout:1586 punch from 0xc75cc00 to 0xc76b400, (0xe800 bytes) 2024-09-11T09:29:15.244 INFO:teuthology.orchestra.run.smithi107.stdout:1587 read 0xc8ac800 thru 0xc8b3692 (0x6e93 bytes) 2024-09-11T09:29:15.244 INFO:teuthology.orchestra.run.smithi107.stdout:1588 punch from 0x9374800 to 0x9379600, (0x4e00 bytes) 2024-09-11T09:29:15.244 INFO:teuthology.orchestra.run.smithi107.stdout:1589 punch from 0x83fae00 to 0x8409c00, (0xee00 bytes) 2024-09-11T09:29:15.244 INFO:teuthology.orchestra.run.smithi107.stdout:1590 write 0xc582600 thru 0xc585fba (0x39bb bytes) 2024-09-11T09:29:15.245 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:15.251 INFO:teuthology.orchestra.run.smithi107.stdout:1593 trunc from 0xdec8000 to 0x76c7800 2024-09-11T09:29:15.272 INFO:teuthology.orchestra.run.smithi107.stdout:1594 read 0x6a94000 thru 0x6aa138b (0xd38c bytes) 2024-09-11T09:29:15.273 INFO:teuthology.orchestra.run.smithi107.stdout:1595 punch from 0x3f46200 to 0x3f53c00, (0xda00 bytes) 2024-09-11T09:29:15.278 INFO:teuthology.orchestra.run.smithi107.stdout:1597 trunc from 0x76c7800 to 0xa5e0c00 2024-09-11T09:29:15.294 INFO:teuthology.orchestra.run.smithi107.stdout:1601 trunc from 0xa5e0c00 to 0x6a9ee00 2024-09-11T09:29:15.295 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:15.296+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:15.309 INFO:teuthology.orchestra.run.smithi107.stdout:1603 trunc from 0x6a9ee00 to 0x5812e00 2024-09-11T09:29:15.325 INFO:teuthology.orchestra.run.smithi107.stdout:1605 punch from 0x14c1200 to 0x14c2200, (0x1000 bytes) 2024-09-11T09:29:15.344 INFO:teuthology.orchestra.run.smithi107.stdout:1606 write 0xcd70000 thru 0xcd7183b (0x183c bytes) 2024-09-11T09:29:15.345 INFO:teuthology.orchestra.run.smithi107.stdout:1608 write 0x4f99600 thru 0x4fa2a5b (0x945c bytes) 2024-09-11T09:29:15.345 INFO:teuthology.orchestra.run.smithi107.stdout:1609 write 0x8a84e00 thru 0x8a93983 (0xeb84 bytes) 2024-09-11T09:29:15.345 INFO:teuthology.orchestra.run.smithi107.stdout:1611 read 0x7345800 thru 0x734ea2c (0x922d bytes) 2024-09-11T09:29:15.346 INFO:teuthology.orchestra.run.smithi107.stdout:1613 read 0x9a2ae00 thru 0x9a2bd89 (0xf8a bytes) 2024-09-11T09:29:15.346 INFO:teuthology.orchestra.run.smithi107.stdout:1614 write 0x8245200 thru 0x82459a4 (0x7a5 bytes) 2024-09-11T09:29:15.346 INFO:teuthology.orchestra.run.smithi107.stdout:1615 punch from 0x3f96a00 to 0x3f97600, (0xc00 bytes) 2024-09-11T09:29:15.351 INFO:teuthology.orchestra.run.smithi107.stdout:1620 trunc from 0xcd71a00 to 0xacdd400 2024-09-11T09:29:15.357 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:15.357 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.990125538999999, 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:15.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:15.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.362 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.990088154, 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:15.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:15.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.990068836000001, 2024-09-11T09:29:15.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:15.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.990057589999999, 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.372 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:15.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:15.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.990015684999999, 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:15.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.381 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989998281, 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:15.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989980385999999, 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:15.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989960205999999, 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:15.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:15.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989939222, 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:15.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989912125, 2024-09-11T09:29:15.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:15.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989892837999999, 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:15.407 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:15.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989688478, 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:15.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989154722, 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.417 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:15.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989105829, 2024-09-11T09:29:15.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:15.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:15.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989057741, 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:15.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.989019703, 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.431 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:15.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.988795861, 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:15.435 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:15.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:15.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:15.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.988618759, 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.440 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:15.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.444 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.988473204, 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.445 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:15.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 11.988351818, 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:15.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:15.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.454 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:15.456 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:15.456+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:15.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1621 trunc from 0xacdd400 to 0xbb38800 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1623 punch from 0x273e600 to 0x274a000, (0xba00 bytes) 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1624 read 0x1607200 thru 0x160c4e6 (0x52e7 bytes) 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1625 read 0xf8fa00 thru 0xf94927 (0x4f28 bytes) 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1626 write 0x762dc00 thru 0x763030e (0x270f bytes) 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1628 trunc from 0xbb38800 to 0xe3b4800 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1629 trunc from 0xe3b4800 to 0x87eca00 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1630 write 0x10da600 thru 0x10dec39 (0x463a bytes) 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1631 read 0x2b82e00 thru 0x2b86405 (0x3606 bytes) 2024-09-11T09:29:15.458 INFO:teuthology.orchestra.run.smithi107.stdout:1632 read 0x70e7000 thru 0x70ebec8 (0x4ec9 bytes) 2024-09-11T09:29:15.459 INFO:teuthology.orchestra.run.smithi107.stdout:1633 read 0x4d37c00 thru 0x4d4309c (0xb49d bytes) 2024-09-11T09:29:15.459 INFO:teuthology.orchestra.run.smithi107.stdout:1634 write 0xe672400 thru 0xe676ec7 (0x4ac8 bytes) 2024-09-11T09:29:15.461 INFO:teuthology.orchestra.run.smithi107.stdout:1636 trunc from 0xe677000 to 0x26f9e00 2024-09-11T09:29:15.487 INFO:teuthology.orchestra.run.smithi107.stdout:1638 read 0xc3c600 thru 0xc4a18c (0xdb8d bytes) 2024-09-11T09:29:15.487 INFO:teuthology.orchestra.run.smithi107.stdout:1640 read 0x2681400 thru 0x268bbd6 (0xa7d7 bytes) 2024-09-11T09:29:15.509 INFO:teuthology.orchestra.run.smithi107.stdout:1641 write 0x85a8e00 thru 0x85ab474 (0x2675 bytes) 2024-09-11T09:29:15.510 INFO:teuthology.orchestra.run.smithi107.stdout:1642 read 0x3271800 thru 0x32810cd (0xf8ce bytes) 2024-09-11T09:29:15.510 INFO:teuthology.orchestra.run.smithi107.stdout:1643 write 0x3249e00 thru 0x32564c7 (0xc6c8 bytes) 2024-09-11T09:29:15.510 INFO:teuthology.orchestra.run.smithi107.stdout:1644 punch from 0x8033200 to 0x803ac00, (0x7a00 bytes) 2024-09-11T09:29:15.510 INFO:teuthology.orchestra.run.smithi107.stdout:1645 read 0x2510400 thru 0x2512384 (0x1f85 bytes) 2024-09-11T09:29:15.510 INFO:teuthology.orchestra.run.smithi107.stdout:1646 write 0x1c64600 thru 0x1c70016 (0xba17 bytes) 2024-09-11T09:29:15.517 INFO:teuthology.orchestra.run.smithi107.stdout:1650 trunc from 0x85ab600 to 0x27c6e00 2024-09-11T09:29:15.551 INFO:teuthology.orchestra.run.smithi107.stdout:1651 trunc from 0x27c6e00 to 0xc75c400 2024-09-11T09:29:15.557 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:15.557+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:15.561 INFO:teuthology.orchestra.run.smithi107.stdout:1652 read 0x4a5dc00 thru 0x4a6a5a1 (0xc9a2 bytes) 2024-09-11T09:29:15.562 INFO:teuthology.orchestra.run.smithi107.stdout:1654 read 0x9ad9800 thru 0x9ae722a (0xda2b bytes) 2024-09-11T09:29:15.562 INFO:teuthology.orchestra.run.smithi107.stdout:1655 read 0x7d67e00 thru 0x7d71710 (0x9911 bytes) 2024-09-11T09:29:15.562 INFO:teuthology.orchestra.run.smithi107.stdout:1656 read 0x6252c00 thru 0x625a95f (0x7d60 bytes) 2024-09-11T09:29:15.563 INFO:teuthology.orchestra.run.smithi107.stdout:1657 trunc from 0xc75c400 to 0xb9b3c00 2024-09-11T09:29:15.567 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:15.567 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:15.567 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:15.576 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:15.581 INFO:teuthology.orchestra.run.smithi107.stdout:1659 punch from 0x52fb600 to 0x5305200, (0x9c00 bytes) 2024-09-11T09:29:15.595 INFO:teuthology.orchestra.run.smithi107.stdout:1663 write 0xce13000 thru 0xce1fbb6 (0xcbb7 bytes) 2024-09-11T09:29:15.600 INFO:teuthology.orchestra.run.smithi107.stdout:1665 trunc from 0xce1fc00 to 0xa15f800 2024-09-11T09:29:15.628 INFO:teuthology.orchestra.run.smithi107.stdout:1668 trunc from 0xa15f800 to 0x5561000 2024-09-11T09:29:15.647 INFO:teuthology.orchestra.run.smithi107.stdout:1669 trunc from 0x5561000 to 0xb8e3600 2024-09-11T09:29:15.657 INFO:teuthology.orchestra.run.smithi107.stdout:1670 trunc from 0xb8e3600 to 0xbe9a400 2024-09-11T09:29:15.658 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:15.658+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:15.682 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:15.682 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:15.683 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:15.683 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:15.689 INFO:teuthology.orchestra.run.smithi107.stdout:1671 write 0xec14c00 thru 0xec1eae4 (0x9ee5 bytes) 2024-09-11T09:29:15.691 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:15.695 INFO:teuthology.orchestra.run.smithi107.stdout:1672 write 0x7ab7a00 thru 0x7ac6226 (0xe827 bytes) 2024-09-11T09:29:15.696 INFO:teuthology.orchestra.run.smithi107.stdout:1674 write 0x2074200 thru 0x20752ff (0x1100 bytes) 2024-09-11T09:29:15.696 INFO:teuthology.orchestra.run.smithi107.stdout:1675 write 0xbadce00 thru 0xbae2f8a (0x618b bytes) 2024-09-11T09:29:15.711 INFO:teuthology.orchestra.run.smithi107.stdout:1677 trunc from 0xec1ec00 to 0x1826e00 2024-09-11T09:29:15.747 INFO:teuthology.orchestra.run.smithi107.stdout:1678 write 0x1331a00 thru 0x133f2b7 (0xd8b8 bytes) 2024-09-11T09:29:15.748 INFO:teuthology.orchestra.run.smithi107.stdout:1679 read 0xcd0e00 thru 0xcdb98f (0xab90 bytes) 2024-09-11T09:29:15.748 INFO:teuthology.orchestra.run.smithi107.stdout:1680 read 0x1370200 thru 0x13794dd (0x92de bytes) 2024-09-11T09:29:15.758 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:15.759+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:15.763 INFO:teuthology.orchestra.run.smithi107.stdout:1681 write 0x4097600 thru 0x40a71d9 (0xfbda bytes) 2024-09-11T09:29:15.764 INFO:teuthology.orchestra.run.smithi107.stdout:1682 read 0x1fb4200 thru 0x1fc4096 (0xfe97 bytes) 2024-09-11T09:29:15.764 INFO:teuthology.orchestra.run.smithi107.stdout:1683 read 0x15aca00 thru 0x15b87c3 (0xbdc4 bytes) 2024-09-11T09:29:15.764 INFO:teuthology.orchestra.run.smithi107.stdout:1685 write 0x2dde00 thru 0x2ebc1a (0xde1b bytes) 2024-09-11T09:29:15.765 INFO:teuthology.orchestra.run.smithi107.stdout:1686 read 0xc5c800 thru 0xc6677e (0x9f7f bytes) 2024-09-11T09:29:15.765 INFO:teuthology.orchestra.run.smithi107.stdout:1688 punch from 0x2091c00 to 0x209fa00, (0xde00 bytes) 2024-09-11T09:29:15.765 INFO:teuthology.orchestra.run.smithi107.stdout:1690 read 0x2cbe200 thru 0x2cc2218 (0x4019 bytes) 2024-09-11T09:29:15.781 INFO:teuthology.orchestra.run.smithi107.stdout:1692 write 0x8688400 thru 0x8690c20 (0x8821 bytes) 2024-09-11T09:29:15.790 INFO:teuthology.orchestra.run.smithi107.stdout:1693 trunc from 0x8690e00 to 0xecd0200 2024-09-11T09:29:15.802 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:29:15.803 INFO:teuthology.orchestra.run.smithi107.stdout:1697 trunc from 0xecd0200 to 0xcccb800 2024-09-11T09:29:15.804 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4835 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8202479900000004, 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033722624999999999, 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4835 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7800000000000002e-07 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983045+0000", 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.657e-06 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983049+0000", 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3420000000000001e-06 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983052+0000", 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.74e-06 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016669+0000", 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033617247000000003 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016711+0000", 2024-09-11T09:29:15.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1149999999999997e-05 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016766+0000", 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5111000000000001e-05 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:15.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8201921409999997, 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:15.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:15.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:15.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8201703660000002, 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.816 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8201243219999999, 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8201126519999997, 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:15.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.830 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8192392430000002, 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.831 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:15.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8188757679999998, 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:15.835 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:15.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:15.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8188243829999999, 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:15.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:15.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8187886820000001, 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:15.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:15.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8187666450000002, 2024-09-11T09:29:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:15.855 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:15.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:15.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:15.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:15.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:15.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.864 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8187310600000002, 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:15.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8186707169999998, 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:15.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:15.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8186220330000005, 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8185655690000004, 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:15.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8185460139999998, 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:15.885 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:15.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:15.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8184982510000003, 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:15.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:15.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8183845129999998, 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:15.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:15.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8183657689999997, 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:15.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8183267799999996, 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:15.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:15.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.818303673, 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:15.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:15.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:15.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:15.915 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:15.917 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:15.916+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:15.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1698 trunc from 0xcccb800 to 0xdf3a00 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1699 write 0xc1c7200 thru 0xc1c972a (0x252b bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1706 trunc from 0xc1c9800 to 0x6c17a00 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1707 read 0x5c6b000 thru 0x5c7679b (0xb79c bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1708 read 0x6b40600 thru 0x6b43e04 (0x3805 bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1711 trunc from 0x6c17a00 to 0xa02ee00 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1712 read 0x4203a00 thru 0x420a438 (0x6a39 bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1714 punch from 0xf3aa00 to 0xf3be00, (0x1400 bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1716 read 0x287c00 thru 0x28a171 (0x2572 bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1718 read 0x5000400 thru 0x500c070 (0xbc71 bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1720 read 0x9442600 thru 0x9451a63 (0xf464 bytes) 2024-09-11T09:29:15.919 INFO:teuthology.orchestra.run.smithi107.stdout:1721 read 0x8a19800 thru 0x8a1f10c (0x590d bytes) 2024-09-11T09:29:15.920 INFO:teuthology.orchestra.run.smithi107.stdout:1722 read 0x15d9800 thru 0x15e039a (0x6b9b bytes) 2024-09-11T09:29:15.932 INFO:teuthology.orchestra.run.smithi107.stdout:1725 write 0xbc66600 thru 0xbc73d6b (0xd76c bytes) 2024-09-11T09:29:15.933 INFO:teuthology.orchestra.run.smithi107.stdout:1726 read 0x32f4c00 thru 0x32f5292 (0x693 bytes) 2024-09-11T09:29:15.933 INFO:teuthology.orchestra.run.smithi107.stdout:1727 read 0x5d63600 thru 0x5d70fb9 (0xd9ba bytes) 2024-09-11T09:29:15.954 INFO:teuthology.orchestra.run.smithi107.stdout:1728 write 0xde22200 thru 0xde22ee1 (0xce2 bytes) 2024-09-11T09:29:15.955 INFO:teuthology.orchestra.run.smithi107.stdout:1730 write 0x339200 thru 0x34892b (0xf72c bytes) 2024-09-11T09:29:15.955 INFO:teuthology.orchestra.run.smithi107.stdout:1732 write 0x9388000 thru 0x9395b8f (0xdb90 bytes) 2024-09-11T09:29:15.964 INFO:teuthology.orchestra.run.smithi107.stdout:1733 trunc from 0xde23000 to 0x5ef4400 2024-09-11T09:29:15.996 INFO:teuthology.orchestra.run.smithi107.stdout:1734 trunc from 0x5ef4400 to 0x2b48400 2024-09-11T09:29:16.007 INFO:teuthology.orchestra.run.smithi107.stdout:1735 read 0xf5f800 thru 0xf6c30c (0xcb0d bytes) 2024-09-11T09:29:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:1737 read 0x187b400 thru 0x187eec9 (0x3aca bytes) 2024-09-11T09:29:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:1738 read 0x2a5f800 thru 0x2a61316 (0x1b17 bytes) 2024-09-11T09:29:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:1739 read 0x1009000 thru 0x1009eac (0xead bytes) 2024-09-11T09:29:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:1741 read 0x1647400 thru 0x164922b (0x1e2c bytes) 2024-09-11T09:29:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:1742 read 0x275da00 thru 0x2769986 (0xbf87 bytes) 2024-09-11T09:29:16.008 INFO:teuthology.orchestra.run.smithi107.stdout:1743 punch from 0x21ddc00 to 0x21ed400, (0xf800 bytes) 2024-09-11T09:29:16.010 INFO:teuthology.orchestra.run.smithi107.stdout:1744 punch from 0x1288e00 to 0x1289c00, (0xe00 bytes) 2024-09-11T09:29:16.010 INFO:teuthology.orchestra.run.smithi107.stdout:1748 read 0x1721000 thru 0x172d8b8 (0xc8b9 bytes) 2024-09-11T09:29:16.011 INFO:teuthology.orchestra.run.smithi107.stdout:1750 trunc from 0x2b48400 to 0x1e89800 2024-09-11T09:29:16.017 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:16.017+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:16.027 INFO:teuthology.orchestra.run.smithi107.stdout:1751 read 0xe46000 thru 0xe50cd2 (0xacd3 bytes) 2024-09-11T09:29:16.029 INFO:teuthology.orchestra.run.smithi107.stdout:1754 punch from 0xbf4c00 to 0xbffe00, (0xb200 bytes) 2024-09-11T09:29:16.044 INFO:teuthology.orchestra.run.smithi107.stdout:1758 write 0x4ea9800 thru 0x4ead573 (0x3d74 bytes) 2024-09-11T09:29:16.044 INFO:teuthology.orchestra.run.smithi107.stdout:1761 read 0x384bc00 thru 0x384d658 (0x1a59 bytes) 2024-09-11T09:29:16.045 INFO:teuthology.orchestra.run.smithi107.stdout:1762 read 0x19e4a00 thru 0x19e96ae (0x4caf bytes) 2024-09-11T09:29:16.045 INFO:teuthology.orchestra.run.smithi107.stdout:1763 read 0x306f600 thru 0x307cb52 (0xd553 bytes) 2024-09-11T09:29:16.048 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:16.048 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:16.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.048 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4706.0:4600 3.1 3:81e4a4d9:::rbd_data.1106a0ba047e.0000000000000000:f [read 32768~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-11T09:29:16.048 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.049247+0000", 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.000163893, 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017203499999999999, 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4706", 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/157994578", 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4600 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049247+0000", 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049247+0000", 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049248+0000", 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8899999999999999e-07 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049251+0000", 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3050000000000001e-06 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049261+0000", 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6020000000000006e-06 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049265+0000", 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5299999999999998e-06 2024-09-11T09:29:16.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049309+0000", 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4373999999999999e-05 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049356+0000", 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6217999999999997e-05 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.049419+0000", 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3417000000000006e-05 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 1 2024-09-11T09:29:16.053 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:16.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:16.060 INFO:teuthology.orchestra.run.smithi107.stdout:1764 write 0x6937c00 thru 0x693969c (0x1a9d bytes) 2024-09-11T09:29:16.067 INFO:teuthology.orchestra.run.smithi107.stdout:1767 read 0x5879e00 thru 0x587fe0d (0x600e bytes) 2024-09-11T09:29:16.067 INFO:teuthology.orchestra.run.smithi107.stdout:1768 read 0x641800 thru 0x64be3b (0xa63c bytes) 2024-09-11T09:29:16.068 INFO:teuthology.orchestra.run.smithi107.stdout:1769 trunc from 0x6939800 to 0x7d60e00 2024-09-11T09:29:16.086 INFO:teuthology.orchestra.run.smithi107.stdout:1771 trunc from 0x7d60e00 to 0x2b3ac00 2024-09-11T09:29:16.107 INFO:teuthology.orchestra.run.smithi107.stdout:1772 write 0x2571200 thru 0x2573ee3 (0x2ce4 bytes) 2024-09-11T09:29:16.135 INFO:teuthology.orchestra.run.smithi107.stdout:1774 write 0xbc86400 thru 0xbc89a74 (0x3675 bytes) 2024-09-11T09:29:16.136 INFO:teuthology.orchestra.run.smithi107.stdout:1775 read 0x3e21400 thru 0x3e217fe (0x3ff bytes) 2024-09-11T09:29:16.136 INFO:teuthology.orchestra.run.smithi107.stdout:1776 write 0x2d10a00 thru 0x2d1e551 (0xdb52 bytes) 2024-09-11T09:29:16.143 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:16.144+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:16.145 INFO:teuthology.orchestra.run.smithi107.stdout:1777 trunc from 0xbc89c00 to 0x3ca5200 2024-09-11T09:29:16.169 INFO:teuthology.orchestra.run.smithi107.stdout:1779 write 0x3c8b800 thru 0x3c96b58 (0xb359 bytes) 2024-09-11T09:29:16.170 INFO:teuthology.orchestra.run.smithi107.stdout:1781 write 0x376b000 thru 0x376c837 (0x1838 bytes) 2024-09-11T09:29:16.170 INFO:teuthology.orchestra.run.smithi107.stdout:1783 read 0x2cbec00 thru 0x2cc3d41 (0x5142 bytes) 2024-09-11T09:29:16.197 INFO:teuthology.orchestra.run.smithi107.stdout:1784 write 0xb72b200 thru 0xb738ddf (0xdbe0 bytes) 2024-09-11T09:29:16.197 INFO:teuthology.orchestra.run.smithi107.stdout:1785 read 0x9dd600 thru 0x9e717e (0x9b7f bytes) 2024-09-11T09:29:16.198 INFO:teuthology.orchestra.run.smithi107.stdout:1787 read 0x43af600 thru 0x43b8934 (0x9335 bytes) 2024-09-11T09:29:16.207 INFO:teuthology.orchestra.run.smithi107.stdout:1788 trunc from 0xb738e00 to 0x3354200 2024-09-11T09:29:16.219 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:16.219 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:16.219 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:16.219 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:16.219 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:16.220 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:16.219+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:16.231 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:16.235 INFO:teuthology.orchestra.run.smithi107.stdout:1791 punch from 0x1730a00 to 0x1733e00, (0x3400 bytes) 2024-09-11T09:29:16.235 INFO:teuthology.orchestra.run.smithi107.stdout:1793 read 0x1b83600 thru 0x1b88887 (0x5288 bytes) 2024-09-11T09:29:16.235 INFO:teuthology.orchestra.run.smithi107.stdout:1796 read 0xd0a000 thru 0xd0db95 (0x3b96 bytes) 2024-09-11T09:29:16.235 INFO:teuthology.orchestra.run.smithi107.stdout:1798 read 0x16ec400 thru 0x16f4802 (0x8403 bytes) 2024-09-11T09:29:16.246 INFO:teuthology.orchestra.run.smithi107.stdout:1800 trunc from 0x3354200 to 0xc8fa400 2024-09-11T09:29:16.262 INFO:teuthology.orchestra.run.smithi107.stdout:1802 write 0xc388a00 thru 0xc38c70b (0x3d0c bytes) 2024-09-11T09:29:16.262 INFO:teuthology.orchestra.run.smithi107.stdout:1804 write 0x901aa00 thru 0x901c9d2 (0x1fd3 bytes) 2024-09-11T09:29:16.262 INFO:teuthology.orchestra.run.smithi107.stdout:1805 read 0x20a0000 thru 0x20ac25a (0xc25b bytes) 2024-09-11T09:29:16.262 INFO:teuthology.orchestra.run.smithi107.stdout:1806 write 0x82f6400 thru 0x8304750 (0xe351 bytes) 2024-09-11T09:29:16.290 INFO:teuthology.orchestra.run.smithi107.stdout:1809 write 0xe287600 thru 0xe29356f (0xbf70 bytes) 2024-09-11T09:29:16.291 INFO:teuthology.orchestra.run.smithi107.stdout:1810 read 0x5009600 thru 0x500c913 (0x3314 bytes) 2024-09-11T09:29:16.294 INFO:teuthology.orchestra.run.smithi107.stdout:1813 trunc from 0xe293600 to 0xae71a00 2024-09-11T09:29:16.320 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:16.320+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:16.322 INFO:teuthology.orchestra.run.smithi107.stdout:1817 read 0x252c400 thru 0x25303d4 (0x3fd5 bytes) 2024-09-11T09:29:16.324 INFO:teuthology.orchestra.run.smithi107.stdout:1819 read 0xad6ae00 thru 0xad6d293 (0x2494 bytes) 2024-09-11T09:29:16.324 INFO:teuthology.orchestra.run.smithi107.stdout:1820 read 0x160be00 thru 0x1617f0c (0xc10d bytes) 2024-09-11T09:29:16.329 INFO:teuthology.orchestra.run.smithi107.stdout:1823 trunc from 0xae71a00 to 0x6968c00 2024-09-11T09:29:16.352 INFO:teuthology.orchestra.run.smithi107.stdout:1824 read 0x3bfd800 thru 0x3c05c30 (0x8431 bytes) 2024-09-11T09:29:16.358 INFO:teuthology.orchestra.run.smithi107.stdout:1825 trunc from 0x6968c00 to 0x1559800 2024-09-11T09:29:16.393 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:16.393 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:16.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:16.393 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.721587225, 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:16.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:16.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:16.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1715 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516363561, 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022344029000000001, 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.400 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1715 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.96e-07 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876407+0000", 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.871e-06 2024-09-11T09:29:16.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876414+0000", 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4679999999999997e-06 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876419+0000", 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1939999999999997e-06 2024-09-11T09:29:16.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898563+0000", 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022144646 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898643+0000", 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9900000000000004e-05 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898749+0000", 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105554 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1716 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516343299999999, 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022552065999999999, 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.405 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1716 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876426+0000", 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.77e-07 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876432+0000", 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0079999999999999e-06 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876438+0000", 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4999999999999996e-06 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876443+0000", 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.259e-06 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898781+0000", 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022337308 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898875+0000", 2024-09-11T09:29:16.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4388000000000005e-05 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898977+0000", 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010222599999999999 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1717 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.410 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516318395000001, 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022753961, 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.411 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1717 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876451+0000", 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876452+0000", 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.905e-06 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876459+0000", 2024-09-11T09:29:16.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4980000000000003e-06 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876464+0000", 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1909999999999999e-06 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899008+0000", 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022544281999999999 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899093+0000", 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4505000000000005e-05 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899204+0000", 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111049 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.415 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1718 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516297731, 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02296196, 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1718 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.416 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876473+0000", 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9259999999999999e-06 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876480+0000", 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876485+0000", 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2159999999999997e-06 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899235+0000", 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022750631 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899321+0000", 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5958999999999995e-05 2024-09-11T09:29:16.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899433+0000", 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011126899999999999 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1719 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 262144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.420 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516257314000001, 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023149355, 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1719 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.421 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876512+0000", 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.44e-07 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876514+0000", 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876522+0000", 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5000000000000002e-06 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876527+0000", 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.587e-06 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899467+0000", 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022939975000000001 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899552+0000", 2024-09-11T09:29:16.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5131999999999997e-05 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899660+0000", 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108137 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1720 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.425 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516233723999999, 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023387654000000001, 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1720 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.63e-07 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876537+0000", 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1040000000000001e-06 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876544+0000", 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7800000000000003e-06 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876549+0000", 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2549999999999997e-06 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899706+0000", 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023156708000000002 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899801+0000", 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4826000000000003e-05 2024-09-11T09:29:16.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899922+0000", 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121518 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1760 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 12288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:16.430 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516212832000001, 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023618561999999999, 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1760 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876558+0000", 2024-09-11T09:29:16.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1670000000000002e-06 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876566+0000", 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3810000000000001e-06 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876571+0000", 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3029999999999999e-06 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899963+0000", 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023392445000000001 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900061+0000", 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7495000000000001e-05 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900174+0000", 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113391 2024-09-11T09:29:16.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1761 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516191032, 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023824222999999999, 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.435 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1761 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:16.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876578+0000", 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876580+0000", 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9539999999999998e-06 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876586+0000", 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876591+0000", 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900220+0000", 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023628546 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900317+0000", 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7168000000000002e-05 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900402+0000", 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4701000000000001e-05 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1762 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 57344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516170776999999, 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024033523000000001, 2024-09-11T09:29:16.440 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1762 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:16.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876600+0000", 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.052e-06 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876607+0000", 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876612+0000", 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9259999999999999e-06 2024-09-11T09:29:16.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900448+0000", 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023836570000000001 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900534+0000", 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5314000000000003e-05 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900631+0000", 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7676000000000005e-05 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1763 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 79872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516150351, 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024166084000000001, 2024-09-11T09:29:16.445 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1763 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876619+0000", 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6400000000000003e-07 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876621+0000", 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0949999999999998e-06 2024-09-11T09:29:16.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876629+0000", 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8380000000000008e-06 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876635+0000", 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3199999999999999e-06 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900675+0000", 2024-09-11T09:29:16.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024040611 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900724+0000", 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8097999999999999e-05 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900784+0000", 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0658000000000001e-05 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.449 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1764 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516127114, 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024356692999999999, 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.450 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1764 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876642+0000", 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876644+0000", 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.041e-06 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876651+0000", 2024-09-11T09:29:16.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6309999999999997e-06 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876656+0000", 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999999e-06 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900814+0000", 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024158129 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900906+0000", 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2618e-05 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900998+0000", 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1624000000000002e-05 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1765 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.454 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516106384, 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024574166000000001, 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1765 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:16.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.39e-07 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876664+0000", 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9520000000000001e-06 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876671+0000", 2024-09-11T09:29:16.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3670000000000002e-06 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876676+0000", 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901029+0000", 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024353631000000001 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901109+0000", 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9951999999999996e-05 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901236+0000", 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012697000000000001 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516086595000001, 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.460 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:16.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516066363, 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.466 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:16.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.470 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516040237, 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.471 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:16.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:16.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.516017952, 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.476 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.477 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:16.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.477 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:16.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.479 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:16.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.482 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:16.482 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:16.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.482 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:16.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:16.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.484 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.484 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:16.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.515994512000001, 2024-09-11T09:29:16.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.486 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:16.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:16.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.515974665, 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:16.490 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:16.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 10.515942288, 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.495 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:16.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:16.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.500 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:16.501 INFO:teuthology.orchestra.run.smithi107.stdout:1833 write 0x17ee600 thru 0x17f7059 (0x8a5a bytes) 2024-09-11T09:29:16.502 INFO:teuthology.orchestra.run.smithi107.stdout:1834 write 0x799b800 thru 0x79a3f86 (0x8787 bytes) 2024-09-11T09:29:16.502 INFO:teuthology.orchestra.run.smithi107.stdout:1836 trunc from 0x79a4000 to 0x5d1600 2024-09-11T09:29:16.502 INFO:teuthology.orchestra.run.smithi107.stdout:1838 write 0x8a4be00 thru 0x8a5166a (0x586b bytes) 2024-09-11T09:29:16.502 INFO:teuthology.orchestra.run.smithi107.stdout:1843 trunc from 0x8a51800 to 0x4bb2400 2024-09-11T09:29:16.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:16.503 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:16.501+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:16.529 INFO:teuthology.orchestra.run.smithi107.stdout:1847 write 0xaa6fc00 thru 0xaa7bd58 (0xc159 bytes) 2024-09-11T09:29:16.529 INFO:teuthology.orchestra.run.smithi107.stdout:1849 write 0x5a5d200 thru 0x5a62864 (0x5665 bytes) 2024-09-11T09:29:16.531 INFO:teuthology.orchestra.run.smithi107.stdout:1850 trunc from 0xaa7be00 to 0x9878000 2024-09-11T09:29:16.559 INFO:teuthology.orchestra.run.smithi107.stdout:1852 write 0x204f800 thru 0x2053dab (0x45ac bytes) 2024-09-11T09:29:16.560 INFO:teuthology.orchestra.run.smithi107.stdout:1853 read 0x1426400 thru 0x1432852 (0xc453 bytes) 2024-09-11T09:29:16.560 INFO:teuthology.orchestra.run.smithi107.stdout:1855 punch from 0x8f1a400 to 0x8f22400, (0x8000 bytes) 2024-09-11T09:29:16.560 INFO:teuthology.orchestra.run.smithi107.stdout:1856 write 0x14eda00 thru 0x14eed0e (0x130f bytes) 2024-09-11T09:29:16.560 INFO:teuthology.orchestra.run.smithi107.stdout:1857 read 0x46b7400 thru 0x46befa4 (0x7ba5 bytes) 2024-09-11T09:29:16.560 INFO:teuthology.orchestra.run.smithi107.stdout:1859 punch from 0x8ba1600 to 0x8ba8800, (0x7200 bytes) 2024-09-11T09:29:16.568 INFO:teuthology.orchestra.run.smithi107.stdout:1860 trunc from 0x9878000 to 0x2135e00 2024-09-11T09:29:16.602 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:16.602+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:16.607 INFO:teuthology.orchestra.run.smithi107.stdout:1862 write 0x26dc600 thru 0x26e82c8 (0xbcc9 bytes) 2024-09-11T09:29:16.608 INFO:teuthology.orchestra.run.smithi107.stdout:1863 punch from 0x14fae00 to 0x1505200, (0xa400 bytes) 2024-09-11T09:29:16.632 INFO:teuthology.orchestra.run.smithi107.stdout:1864 write 0x99a0e00 thru 0x99a12de (0x4df bytes) 2024-09-11T09:29:16.653 INFO:teuthology.orchestra.run.smithi107.stdout:1865 write 0xdca6400 thru 0xdcb392d (0xd52e bytes) 2024-09-11T09:29:16.653 INFO:teuthology.orchestra.run.smithi107.stdout:1866 read 0x5500600 thru 0x550b51a (0xaf1b bytes) 2024-09-11T09:29:16.653 INFO:teuthology.orchestra.run.smithi107.stdout:1868 write 0x22ec200 thru 0x22fb94b (0xf74c bytes) 2024-09-11T09:29:16.653 INFO:teuthology.orchestra.run.smithi107.stdout:1869 read 0x158b200 thru 0x15921c9 (0x6fca bytes) 2024-09-11T09:29:16.653 INFO:teuthology.orchestra.run.smithi107.stdout:1871 punch from 0x27cf200 to 0x27d1600, (0x2400 bytes) 2024-09-11T09:29:16.654 INFO:teuthology.orchestra.run.smithi107.stdout:1872 read 0x3452c00 thru 0x345e0ce (0xb4cf bytes) 2024-09-11T09:29:16.654 INFO:teuthology.orchestra.run.smithi107.stdout:1873 write 0x8977c00 thru 0x897be43 (0x4244 bytes) 2024-09-11T09:29:16.654 INFO:teuthology.orchestra.run.smithi107.stdout:1875 read 0xc38600 thru 0xc38792 (0x193 bytes) 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4706.0:6021 3.0 3:0130708f:::rbd_object_map.110647be545c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.654885+0000", 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.0049957489999999998, 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0050194549999999999, 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.659 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for sub ops", 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4706", 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/157994578", 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 6021 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654885+0000", 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654885+0000", 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654885+0000", 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654888+0000", 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6869999999999999e-06 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654893+0000", 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2989999999999996e-06 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654897+0000", 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4649999999999999e-06 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.656244+0000", 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013474629999999999 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.656335+0000", 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0946000000000006e-05 2024-09-11T09:29:16.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.656741+0000", 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00040538500000000002 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658558+0000", 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018179369999999999 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4706.0:6022 3.0 3:0130708f:::rbd_object_map.110647be545c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.654907+0000", 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.0049736370000000004, 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0050500379999999997, 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for sub ops", 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4706", 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/157994578", 2024-09-11T09:29:16.665 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 6022 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654907+0000", 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654907+0000", 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654907+0000", 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1e-07 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654909+0000", 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654913+0000", 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.692e-06 2024-09-11T09:29:16.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.654916+0000", 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657589+0000", 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0026729229999999998 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657639+0000", 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0853999999999998e-05 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658036+0000", 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00039663699999999998 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4706.0:6024 3.0 3:0130708f:::rbd_object_map.110647be545c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-11T09:29:16.669 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.655072+0000", 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.0048088710000000002, 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0049260349999999996, 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for sub ops", 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4706", 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/157994578", 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 6024 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655072+0000", 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655072+0000", 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655072+0000", 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:16.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655074+0000", 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2699999999999999e-06 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655078+0000", 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9110000000000003e-06 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655082+0000", 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3450000000000002e-06 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658590+0000", 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0035087920000000002 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658674+0000", 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4027e-05 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658998+0000", 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000323882 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4706.0:6025 3.0 3:0130708f:::rbd_object_map.110647be545c:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.655089+0000", 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.0047914450000000001, 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0049486039999999997, 2024-09-11T09:29:16.674 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for sub ops", 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4706", 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/157994578", 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 6025 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655089+0000", 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655089+0000", 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655089+0000", 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4999999999999999e-07 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655091+0000", 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.686e-06 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655095+0000", 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.596e-06 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.655097+0000", 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659352+0000", 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0042547000000000001 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659441+0000", 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9085000000000005e-05 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659821+0000", 2024-09-11T09:29:16.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00038020099999999999 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4706.0:6033 3.0 3.302c580 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.657658+0000", 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.0022227739999999998, 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.002416381, 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "queued for pg", 2024-09-11T09:29:16.679 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4706", 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/157994578", 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 6033 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657658+0000", 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657658+0000", 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657659+0000", 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.031e-06 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657674+0000", 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5628e-05 2024-09-11T09:29:16.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657688+0000", 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3918e-05 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.657698+0000", 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.312e-06 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4706.0:6022 3.0 e43/26)", 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.658809+0000", 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.001071862, 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001291997, 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "queued for pg", 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658809+0000", 2024-09-11T09:29:16.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658809+0000", 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658809+0000", 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.16e-07 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658817+0000", 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7239999999999992e-06 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658835+0000", 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7779999999999999e-05 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.658843+0000", 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4509999999999992e-06 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.685 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4706.0:6024 3.0 e43/26)", 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:16.659700+0000", 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.00018026699999999999, 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00042302000000000003, 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "queued for pg", 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659700+0000", 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659700+0000", 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659701+0000", 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:29:16.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659704+0000", 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.18e-06 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659714+0000", 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8670000000000006e-06 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:16.659719+0000", 2024-09-11T09:29:16.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8119999999999998e-06 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 7 2024-09-11T09:29:16.689 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:16.690 INFO:teuthology.orchestra.run.smithi107.stdout:1877 trunc from 0xdcb3a00 to 0x4615c00 2024-09-11T09:29:16.690 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:16.696 INFO:teuthology.orchestra.run.smithi107.stdout:1879 trunc from 0x4615c00 to 0x2ceb400 2024-09-11T09:29:16.711 INFO:teuthology.orchestra.run.smithi107.stdout:1880 read 0xd73a00 thru 0xd7db4d (0xa14e bytes) 2024-09-11T09:29:16.711 INFO:teuthology.orchestra.run.smithi107.stdout:1882 read 0x1cab200 thru 0x1caf783 (0x4584 bytes) 2024-09-11T09:29:16.731 INFO:teuthology.orchestra.run.smithi107.stdout:1883 write 0x80cf400 thru 0x80de262 (0xee63 bytes) 2024-09-11T09:29:16.732 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:16.732+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:16.750 INFO:teuthology.orchestra.run.smithi107.stdout:1884 write 0x8719000 thru 0x8719198 (0x199 bytes) 2024-09-11T09:29:16.750 INFO:teuthology.orchestra.run.smithi107.stdout:1885 write 0x732ae00 thru 0x73304dc (0x56dd bytes) 2024-09-11T09:29:16.750 INFO:teuthology.orchestra.run.smithi107.stdout:1886 write 0xf8d000 thru 0xf960d3 (0x90d4 bytes) 2024-09-11T09:29:16.750 INFO:teuthology.orchestra.run.smithi107.stdout:1887 punch from 0x1695800 to 0x169cc00, (0x7400 bytes) 2024-09-11T09:29:16.750 INFO:teuthology.orchestra.run.smithi107.stdout:1888 punch from 0x14f4c00 to 0x14f7a00, (0x2e00 bytes) 2024-09-11T09:29:16.750 INFO:teuthology.orchestra.run.smithi107.stdout:1889 write 0x4b03200 thru 0x4b09a27 (0x6828 bytes) 2024-09-11T09:29:16.751 INFO:teuthology.orchestra.run.smithi107.stdout:1890 trunc from 0x8719200 to 0x8eaa400 2024-09-11T09:29:16.767 INFO:teuthology.orchestra.run.smithi107.stdout:1891 read 0x8c88600 thru 0x8c8f3ee (0x6def bytes) 2024-09-11T09:29:16.768 INFO:teuthology.orchestra.run.smithi107.stdout:1892 punch from 0xe6de00 to 0xe72600, (0x4800 bytes) 2024-09-11T09:29:16.783 INFO:teuthology.orchestra.run.smithi107.stdout:1893 write 0xd055000 thru 0xd059fcb (0x4fcc bytes) 2024-09-11T09:29:16.786 INFO:teuthology.orchestra.run.smithi107.stdout:1895 trunc from 0xd05a000 to 0xb4a1400 2024-09-11T09:29:16.795 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:29:16.803 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:16.804+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:16.810 INFO:teuthology.orchestra.run.smithi107.stdout:1896 trunc from 0xb4a1400 to 0x7c5f600 2024-09-11T09:29:16.825 INFO:teuthology.orchestra.run.smithi107.stdout:1899 write 0x6844e00 thru 0x68546fa (0xf8fb bytes) 2024-09-11T09:29:16.826 INFO:teuthology.orchestra.run.smithi107.stdout:1900 read 0x6def200 thru 0x6df7f68 (0x8d69 bytes) 2024-09-11T09:29:16.833 INFO:teuthology.orchestra.run.smithi107.stdout:1901 trunc from 0x7c5f600 to 0xe661000 2024-09-11T09:29:16.847 INFO:teuthology.orchestra.run.smithi107.stdout:1902 write 0x2e11200 thru 0x2e1cf33 (0xbd34 bytes) 2024-09-11T09:29:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:1906 write 0x608d600 thru 0x609cea7 (0xf8a8 bytes) 2024-09-11T09:29:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:1908 read 0xbaae800 thru 0xbab739d (0x8b9e bytes) 2024-09-11T09:29:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:1909 write 0x1f7bc00 thru 0x1f8b3df (0xf7e0 bytes) 2024-09-11T09:29:16.857 INFO:teuthology.orchestra.run.smithi107.stdout:1910 trunc from 0xe661000 to 0x731a400 2024-09-11T09:29:16.863 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:16.863 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:16.863 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:16.863 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:16.863 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:16.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:16.877 INFO:teuthology.orchestra.run.smithi107.stdout:1911 write 0x5ca3600 thru 0x5cab1a3 (0x7ba4 bytes) 2024-09-11T09:29:16.896 INFO:teuthology.orchestra.run.smithi107.stdout:1912 write 0xc97e200 thru 0xc98393a (0x573b bytes) 2024-09-11T09:29:16.896 INFO:teuthology.orchestra.run.smithi107.stdout:1916 write 0x4578600 thru 0x457b643 (0x3044 bytes) 2024-09-11T09:29:16.896 INFO:teuthology.orchestra.run.smithi107.stdout:1917 punch from 0x27f7000 to 0x27fdc00, (0x6c00 bytes) 2024-09-11T09:29:16.901 INFO:teuthology.orchestra.run.smithi107.stdout:1920 read 0xa0cd000 thru 0xa0dadd5 (0xddd6 bytes) 2024-09-11T09:29:16.901 INFO:teuthology.orchestra.run.smithi107.stdout:1922 punch from 0xa275600 to 0xa275a00, (0x400 bytes) 2024-09-11T09:29:16.901 INFO:teuthology.orchestra.run.smithi107.stdout:1923 write 0x47bec00 thru 0x47c4b36 (0x5f37 bytes) 2024-09-11T09:29:16.901 INFO:teuthology.orchestra.run.smithi107.stdout:1924 punch from 0xa96a600 to 0xa970000, (0x5a00 bytes) 2024-09-11T09:29:16.901 INFO:teuthology.orchestra.run.smithi107.stdout:1925 punch from 0x6628400 to 0x662f400, (0x7000 bytes) 2024-09-11T09:29:16.917 INFO:teuthology.orchestra.run.smithi107.stdout:1929 write 0xdcb2600 thru 0xdcbdb1d (0xb51e bytes) 2024-09-11T09:29:16.917 INFO:teuthology.orchestra.run.smithi107.stdout:1930 read 0xdc7be00 thru 0xdc83247 (0x7448 bytes) 2024-09-11T09:29:16.919 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:16.919+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:16.927 INFO:teuthology.orchestra.run.smithi107.stdout:1933 trunc from 0xdcbdc00 to 0x5661400 2024-09-11T09:29:16.965 INFO:teuthology.orchestra.run.smithi107.stdout:1934 read 0x3931200 thru 0x393c6a4 (0xb4a5 bytes) 2024-09-11T09:29:16.965 INFO:teuthology.orchestra.run.smithi107.stdout:1935 write 0x2084400 thru 0x208514f (0xd50 bytes) 2024-09-11T09:29:16.969 INFO:teuthology.orchestra.run.smithi107.stdout:1939 trunc from 0x5661400 to 0x2254e00 2024-09-11T09:29:16.989 INFO:teuthology.orchestra.run.smithi107.stdout:1940 read 0x1149e00 thru 0x1151bf0 (0x7df1 bytes) 2024-09-11T09:29:17.001 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.090725623, 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.002 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:17.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.005 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.006 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0904950510000004, 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:17.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.010 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.011 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0903978470000002, 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:17.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0901630830000002, 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:17.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0899670669999999, 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.021 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:17.025 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.08969863, 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:17.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0895968800000002, 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:17.031 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:17.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:17.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0892032230000002, 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.036 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:17.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.039 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.040 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0890769779999996, 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.041 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:17.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.088867596, 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.045 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.046 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:17.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0887566470000003, 2024-09-11T09:29:17.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:17.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0885007089999998, 2024-09-11T09:29:17.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:17.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:17.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0883481799999997, 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:17.061 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:17.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:17.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.066 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0881558160000004, 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:17.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:17.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0878253979999997, 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.072 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:17.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0877038460000001, 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.077 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:17.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:17.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0875490680000004, 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.082 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.083 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:17.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:17.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0874951480000004, 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.087 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.088 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:17.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0872945789999999, 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:17.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:17.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:17.096 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.0869390399999999, 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:17.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:17.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:17.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:17.101 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:17.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:17.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:17.104 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:17.103+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1943 write 0xc186400 thru 0xc18a1c9 (0x3dca bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1944 punch from 0x7fc400 to 0x803e00, (0x7a00 bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1945 punch from 0xa8dc00 to 0xa99200, (0xb600 bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1946 read 0x94dcc00 thru 0x94e14a7 (0x48a8 bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1947 write 0xdc94800 thru 0xdc976b6 (0x2eb7 bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1949 trunc from 0xdc97800 to 0x17c4e00 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1950 write 0x361fa00 thru 0x3623f42 (0x4543 bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1951 read 0x1385e00 thru 0x138d5c8 (0x77c9 bytes) 2024-09-11T09:29:17.105 INFO:teuthology.orchestra.run.smithi107.stdout:1952 trunc from 0x3624000 to 0x79b4a00 2024-09-11T09:29:17.115 INFO:teuthology.orchestra.run.smithi107.stdout:1953 write 0x4d28400 thru 0x4d2ecec (0x68ed bytes) 2024-09-11T09:29:17.131 INFO:teuthology.orchestra.run.smithi107.stdout:1956 write 0xaab9c00 thru 0xaabe64d (0x4a4e bytes) 2024-09-11T09:29:17.131 INFO:teuthology.orchestra.run.smithi107.stdout:1957 punch from 0x9125e00 to 0x9126400, (0x600 bytes) 2024-09-11T09:29:17.133 INFO:teuthology.orchestra.run.smithi107.stdout:1958 read 0x8541a00 thru 0x854a088 (0x8689 bytes) 2024-09-11T09:29:17.133 INFO:teuthology.orchestra.run.smithi107.stdout:1960 punch from 0x2db7c00 to 0x2db8000, (0x400 bytes) 2024-09-11T09:29:17.134 INFO:teuthology.orchestra.run.smithi107.stdout:1964 write 0x3810a00 thru 0x38123aa (0x19ab bytes) 2024-09-11T09:29:17.134 INFO:teuthology.orchestra.run.smithi107.stdout:1965 punch from 0x58dcc00 to 0x58e5200, (0x8600 bytes) 2024-09-11T09:29:17.136 INFO:teuthology.orchestra.run.smithi107.stdout:1966 punch from 0x85ef800 to 0x85fd400, (0xdc00 bytes) 2024-09-11T09:29:17.141 INFO:teuthology.orchestra.run.smithi107.stdout:1967 trunc from 0xaabe800 to 0x41d2e00 2024-09-11T09:29:17.159 INFO:teuthology.orchestra.run.smithi107.stdout:1968 read 0x1cf0e00 thru 0x1cf1047 (0x248 bytes) 2024-09-11T09:29:17.160 INFO:teuthology.orchestra.run.smithi107.stdout:1969 punch from 0x3025400 to 0x302a400, (0x5000 bytes) 2024-09-11T09:29:17.184 INFO:teuthology.orchestra.run.smithi107.stdout:1971 write 0xee26400 thru 0xee2f468 (0x9069 bytes) 2024-09-11T09:29:17.184 INFO:teuthology.orchestra.run.smithi107.stdout:1972 read 0xa4a9800 thru 0xa4b8fad (0xf7ae bytes) 2024-09-11T09:29:17.185 INFO:teuthology.orchestra.run.smithi107.stdout:1973 punch from 0x9905c00 to 0x9913000, (0xd400 bytes) 2024-09-11T09:29:17.187 INFO:teuthology.orchestra.run.smithi107.stdout:1976 write 0x3113c00 thru 0x311b92d (0x7d2e bytes) 2024-09-11T09:29:17.187 INFO:teuthology.orchestra.run.smithi107.stdout:1977 write 0xfcce00 thru 0xfd6354 (0x9555 bytes) 2024-09-11T09:29:17.187 INFO:teuthology.orchestra.run.smithi107.stdout:1978 write 0xde82200 thru 0xde91a5f (0xf860 bytes) 2024-09-11T09:29:17.187 INFO:teuthology.orchestra.run.smithi107.stdout:1979 read 0x60ae200 thru 0x60ba258 (0xc059 bytes) 2024-09-11T09:29:17.198 INFO:teuthology.orchestra.run.smithi107.stdout:1981 trunc from 0xee2f600 to 0x4c91800 2024-09-11T09:29:17.203 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:17.204+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:17.221 INFO:teuthology.orchestra.run.smithi107.stdout:1985 write 0x2bd1600 thru 0x2bd665e (0x505f bytes) 2024-09-11T09:29:17.225 INFO:teuthology.orchestra.run.smithi107.stdout:1986 trunc from 0x4c91800 to 0x6d98600 2024-09-11T09:29:17.242 INFO:teuthology.orchestra.run.smithi107.stdout:1987 trunc from 0x6d98600 to 0x857cc00 2024-09-11T09:29:17.253 INFO:teuthology.orchestra.run.smithi107.stdout:1988 read 0x43bc800 thru 0x43c23da (0x5bdb bytes) 2024-09-11T09:29:17.253 INFO:teuthology.orchestra.run.smithi107.stdout:1991 trunc from 0x857cc00 to 0x7e58600 2024-09-11T09:29:17.266 INFO:teuthology.orchestra.run.smithi107.stdout:1992 read 0x150ae00 thru 0x151180d (0x6a0e bytes) 2024-09-11T09:29:17.268 INFO:teuthology.orchestra.run.smithi107.stdout:1993 read 0x46a1200 thru 0x46a98bc (0x86bd bytes) 2024-09-11T09:29:17.268 INFO:teuthology.orchestra.run.smithi107.stdout:1994 read 0x7842600 thru 0x784924a (0x6c4b bytes) 2024-09-11T09:29:17.268 INFO:teuthology.orchestra.run.smithi107.stdout:1995 clone 14 order 18 su 2048 sc 5 2024-09-11T09:29:17.276 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:17.276 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:17.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:17.276 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:17.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:17.287 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:17.287 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:17.304 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:17.305 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:17.305 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:17.305+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:17.321 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:17.322 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:17.339 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:17.339 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:17.355 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:17.356 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:17.373 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:17.394 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:17.395 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:17.403 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:17.405 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:17.405+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:17.506 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:17.506+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:17.550 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:17.550 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:17.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:17.607 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:17.608+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:17.672 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:17.715 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:17.716+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:17.775 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:17.775 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:17.784 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:17.824 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:17.825+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:17.859 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone13 from 0x7e58600 (overlap 0x4e9c00) to 0xfb800 2024-09-11T09:29:17.910 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:17.911+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:17.937 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:17.937 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:17.946 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:18.011 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:18.011+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:18.050 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.050 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.059 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:18.142 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:18.143+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:18.170 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-11T09:29:18.206 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.206 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.212 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:18.213+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:18.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:18.313 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:18.314+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:18.369 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.369 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.379 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:18.414 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:18.415+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:18.515 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:18.516+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:18.531 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.531 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:18.616 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:18.617+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:18.684 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.685 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.691 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:18.734 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:18.735+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:18.753 INFO:teuthology.orchestra.run.smithi107.stdout:1997 write 0xa72b400 thru 0xa737259 (0xbe5a bytes) 2024-09-11T09:29:18.754 INFO:teuthology.orchestra.run.smithi107.stdout:1998 read 0x9556000 thru 0x955cb73 (0x6b74 bytes) 2024-09-11T09:29:18.754 INFO:teuthology.orchestra.run.smithi107.stdout:1999 read 0x8b37400 thru 0x8b43c48 (0xc849 bytes) 2024-09-11T09:29:18.754 INFO:teuthology.orchestra.run.smithi107.stdout:2000 punch from 0x33e4a00 to 0x33f1400, (0xca00 bytes) 2024-09-11T09:29:18.758 INFO:teuthology.orchestra.run.smithi107.stdout:2005 trunc from 0xa737400 to 0xb7f7400 2024-09-11T09:29:18.785 INFO:teuthology.orchestra.run.smithi107.stdout:2008 trunc from 0xb7f7400 to 0xca37200 2024-09-11T09:29:18.801 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.801 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:29:18.813 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:18.856 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:18.856+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:18.919 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:18.920+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:18.925 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:18.925 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:18.937 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:19.020 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:19.021+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:19.058 INFO:teuthology.orchestra.run.smithi107.stdout:2009 punch from 0x7f61200 to 0x7f61800, (0x600 bytes) 2024-09-11T09:29:19.059 INFO:teuthology.orchestra.run.smithi107.stdout:2014 punch from 0x167cc00 to 0x1687200, (0xa600 bytes) 2024-09-11T09:29:19.063 INFO:teuthology.orchestra.run.smithi107.stdout:2016 write 0x4cf3400 thru 0x4cff0ca (0xbccb bytes) 2024-09-11T09:29:19.063 INFO:teuthology.orchestra.run.smithi107.stdout:2017 write 0xb6bce00 thru 0xb6c4bb4 (0x7db5 bytes) 2024-09-11T09:29:19.063 INFO:teuthology.orchestra.run.smithi107.stdout:2018 read 0xb3bc600 thru 0xb3c55e2 (0x8fe3 bytes) 2024-09-11T09:29:19.065 INFO:teuthology.orchestra.run.smithi107.stdout:2020 trunc from 0xca37200 to 0xaf5c000 2024-09-11T09:29:19.088 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:19.088 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:19.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:19.115 INFO:teuthology.orchestra.run.smithi107.stdout:2021 punch from 0x5295e00 to 0x52a0400, (0xa600 bytes) 2024-09-11T09:29:19.125 INFO:teuthology.orchestra.run.smithi107.stdout:2022 read 0xed5400 thru 0xee42e9 (0xeeea bytes) 2024-09-11T09:29:19.125 INFO:teuthology.orchestra.run.smithi107.stdout:2024 punch from 0x753ee00 to 0x754ae00, (0xc000 bytes) 2024-09-11T09:29:19.142 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:19.143+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:19.149 INFO:teuthology.orchestra.run.smithi107.stdout:2026 write 0xecf7200 thru 0xecfde90 (0x6c91 bytes) 2024-09-11T09:29:19.150 INFO:teuthology.orchestra.run.smithi107.stdout:2027 write 0x4388a00 thru 0x43980c0 (0xf6c1 bytes) 2024-09-11T09:29:19.150 INFO:teuthology.orchestra.run.smithi107.stdout:2032 read 0x68bea00 thru 0x68bf494 (0xa95 bytes) 2024-09-11T09:29:19.150 INFO:teuthology.orchestra.run.smithi107.stdout:2033 write 0x2198800 thru 0x219d1dc (0x49dd bytes) 2024-09-11T09:29:19.150 INFO:teuthology.orchestra.run.smithi107.stdout:2035 read 0x5bcdc00 thru 0x5bd4460 (0x6861 bytes) 2024-09-11T09:29:19.150 INFO:teuthology.orchestra.run.smithi107.stdout:2039 write 0x3560200 thru 0x35641bb (0x3fbc bytes) 2024-09-11T09:29:19.157 INFO:teuthology.orchestra.run.smithi107.stdout:2044 trunc from 0xecfe000 to 0x8d98a00 2024-09-11T09:29:19.211 INFO:teuthology.orchestra.run.smithi107.stdout:2046 trunc from 0x8d98a00 to 0xba0b000 2024-09-11T09:29:19.222 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:19.222+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:19.225 INFO:teuthology.orchestra.run.smithi107.stdout:2048 read 0x63b2a00 thru 0x63b8d19 (0x631a bytes) 2024-09-11T09:29:19.225 INFO:teuthology.orchestra.run.smithi107.stdout:2050 write 0x6da0400 thru 0x6dae3bf (0xdfc0 bytes) 2024-09-11T09:29:19.253 INFO:teuthology.orchestra.run.smithi107.stdout:2052 write 0xeaf8000 thru 0xeb068a7 (0xe8a8 bytes) 2024-09-11T09:29:19.264 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:19.264 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:19.266 INFO:teuthology.orchestra.run.smithi107.stdout:2054 trunc from 0xeb06a00 to 0x34b9c00 2024-09-11T09:29:19.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:19.321 INFO:teuthology.orchestra.run.smithi107.stdout:2055 punch from 0x250fa00 to 0x251d800, (0xde00 bytes) 2024-09-11T09:29:19.322 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:19.323+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:19.324 INFO:teuthology.orchestra.run.smithi107.stdout:2060 punch from 0x2270000 to 0x2279000, (0x9000 bytes) 2024-09-11T09:29:19.326 INFO:teuthology.orchestra.run.smithi107.stdout:2061 read 0x251b000 thru 0x252733b (0xc33c bytes) 2024-09-11T09:29:19.365 INFO:teuthology.orchestra.run.smithi107.stdout:2062 write 0xec5bc00 thru 0xec6a2cd (0xe6ce bytes) 2024-09-11T09:29:19.365 INFO:teuthology.orchestra.run.smithi107.stdout:2067 write 0xc2e9000 thru 0xc2f3b44 (0xab45 bytes) 2024-09-11T09:29:19.366 INFO:teuthology.orchestra.run.smithi107.stdout:2068 punch from 0x44c9000 to 0x44d4a00, (0xba00 bytes) 2024-09-11T09:29:19.367 INFO:teuthology.orchestra.run.smithi107.stdout:2069 write 0xe838c00 thru 0xe848499 (0xf89a bytes) 2024-09-11T09:29:19.367 INFO:teuthology.orchestra.run.smithi107.stdout:2072 write 0x1967000 thru 0x197421e (0xd21f bytes) 2024-09-11T09:29:19.384 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:19.384 INFO:teuthology.orchestra.run.smithi107.stdout:2073 trunc from 0xec6a400 to 0x759e00 2024-09-11T09:29:19.401 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:19.401 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:19.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:19.423 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:19.423+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:19.475 INFO:teuthology.orchestra.run.smithi107.stdout:2075 write 0x3db1800 thru 0x3dbe0c3 (0xc8c4 bytes) 2024-09-11T09:29:19.479 INFO:teuthology.orchestra.run.smithi107.stdout:2077 trunc from 0x3dbe200 to 0x53a00 2024-09-11T09:29:19.524 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:19.525+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:19.542 INFO:teuthology.orchestra.run.smithi107.stdout:2081 read 0x50800 thru 0x538fd (0x30fe bytes) 2024-09-11T09:29:19.544 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:19.544 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:19.545 INFO:teuthology.orchestra.run.smithi107.stdout:2086 read 0x8c00 thru 0xda71 (0x4e72 bytes) 2024-09-11T09:29:19.546 INFO:teuthology.orchestra.run.smithi107.stdout:2087 read 0x4c00 thru 0xb223 (0x6624 bytes) 2024-09-11T09:29:19.559 INFO:teuthology.orchestra.run.smithi107.stdout:2088 trunc from 0x53a00 to 0xa919000 2024-09-11T09:29:19.562 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:19.562 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:19.568 INFO:teuthology.orchestra.run.smithi107.stdout:2090 punch from 0x6c1d800 to 0x6c28600, (0xae00 bytes) 2024-09-11T09:29:19.570 INFO:teuthology.orchestra.run.smithi107.stdout:2091 read 0x343e00 thru 0x34469b (0x89c bytes) 2024-09-11T09:29:19.570 INFO:teuthology.orchestra.run.smithi107.stdout:2092 read 0x5b60800 thru 0x5b660c1 (0x58c2 bytes) 2024-09-11T09:29:19.570 INFO:teuthology.orchestra.run.smithi107.stdout:2096 read 0x7e1fa00 thru 0x7e2e0d7 (0xe6d8 bytes) 2024-09-11T09:29:19.571 INFO:teuthology.orchestra.run.smithi107.stdout:2099 read 0x1370e00 thru 0x1373b8b (0x2d8c bytes) 2024-09-11T09:29:19.571 INFO:teuthology.orchestra.run.smithi107.stdout:2100 punch from 0x94aa400 to 0x94adc00, (0x3800 bytes) 2024-09-11T09:29:19.584 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:19.584 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:19.585 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:19.585 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:19.585 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:19.585 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:19.585 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:19.591 INFO:teuthology.orchestra.run.smithi107.stdout:2101 write 0xcaad600 thru 0xcab94fe (0xbeff bytes) 2024-09-11T09:29:19.592 INFO:teuthology.orchestra.run.smithi107.stdout:2102 read 0x822de00 thru 0x8235345 (0x7546 bytes) 2024-09-11T09:29:19.593 INFO:teuthology.orchestra.run.smithi107.stdout:2106 read 0xaf37c00 thru 0xaf38971 (0xd72 bytes) 2024-09-11T09:29:19.593 INFO:teuthology.orchestra.run.smithi107.stdout:2110 read 0x6882000 thru 0x68863e6 (0x43e7 bytes) 2024-09-11T09:29:19.594 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:19.599 INFO:teuthology.orchestra.run.smithi107.stdout:2111 trunc from 0xcab9600 to 0x6e8ae00 2024-09-11T09:29:19.602 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:19.602 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:19.622 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:19.623 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:19.625 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:19.625+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:19.627 INFO:teuthology.orchestra.run.smithi107.stdout:2116 punch from 0x1855a00 to 0x1863e00, (0xe400 bytes) 2024-09-11T09:29:19.629 INFO:teuthology.orchestra.run.smithi107.stdout:2117 read 0x5915000 thru 0x5917e5f (0x2e60 bytes) 2024-09-11T09:29:19.629 INFO:teuthology.orchestra.run.smithi107.stdout:2118 read 0x17adc00 thru 0x17b182c (0x3c2d bytes) 2024-09-11T09:29:19.629 INFO:teuthology.orchestra.run.smithi107.stdout:2120 read 0x1c39a00 thru 0x1c4523d (0xb83e bytes) 2024-09-11T09:29:19.629 INFO:teuthology.orchestra.run.smithi107.stdout:2122 punch from 0x5cc4000 to 0x5cd0200, (0xc200 bytes) 2024-09-11T09:29:19.654 INFO:teuthology.orchestra.run.smithi107.stdout:2126 write 0xa751400 thru 0xa753ee0 (0x2ae1 bytes) 2024-09-11T09:29:19.654 INFO:teuthology.orchestra.run.smithi107.stdout:2127 read 0x92ee000 thru 0x92f85ea (0xa5eb bytes) 2024-09-11T09:29:19.656 INFO:teuthology.orchestra.run.smithi107.stdout:2128 trunc from 0xa754000 to 0x8df4a00 2024-09-11T09:29:19.656 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:19.657 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:19.660 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:19.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T09:29:19.690 INFO:teuthology.orchestra.run.smithi107.stdout:2132 trunc from 0x8df4a00 to 0x59cac00 2024-09-11T09:29:19.716 INFO:teuthology.orchestra.run.smithi107.stdout:2136 trunc from 0x59cac00 to 0xcefe200 2024-09-11T09:29:19.724 INFO:teuthology.orchestra.run.smithi107.stdout:2138 read 0x6c6800 thru 0x6d574f (0xef50 bytes) 2024-09-11T09:29:19.725 INFO:teuthology.orchestra.run.smithi107.stdout:2142 punch from 0x5ad2600 to 0x5ad5e00, (0x3800 bytes) 2024-09-11T09:29:19.725 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:19.726+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8175040350000007, 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:19.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:19.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8172734629999994, 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:19.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:19.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.817176259, 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.739 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:19.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:19.743 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.816941495, 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:19.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8167454789999997, 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.748 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:19.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8164770420000007, 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:19.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:19.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.816375292, 2024-09-11T09:29:19.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:19.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.815981635, 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:19.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:19.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8158553899999994, 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:19.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8156460079999999, 2024-09-11T09:29:19.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:19.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:19.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:19.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:19.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8155350590000001, 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:19.778 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8152791209999997, 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:19.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8151265920000004, 2024-09-11T09:29:19.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.793 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8149342280000003, 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:19.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8146038099999995, 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.814482258, 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8143274799999993, 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8142735600000002, 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8140729909999997, 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.8137174520000006, 2024-09-11T09:29:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:19.829 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:19.831 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:19.831+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:19.831 INFO:teuthology.orchestra.run.smithi107.stdout:2148 write 0x893b200 thru 0x894602d (0xae2e bytes) 2024-09-11T09:29:19.831 INFO:teuthology.orchestra.run.smithi107.stdout:2149 clone 15 order 18 su 4096 sc 5 2024-09-11T09:29:19.832 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:29:19.885 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone14 from 0xcefe200 (overlap 0x53a00) to 0x29c00 2024-09-11T09:29:19.932 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:19.932+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:19.985 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:19.986 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:19.986 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:19.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:20.033 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:20.033+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:20.102 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:20.103 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:20.103 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:20.103 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:20.103 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:20.111 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:20.133 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:20.134+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:20.225 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 43.766492675999999, 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:20.226 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:20.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:20.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:20.231 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.504972987999999, 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:20.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:20.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:20.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:20.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:20.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.503727021, 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:20.238 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:20.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:20.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:20.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:20.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:20.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:20.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:20.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:20.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:20.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:20.243 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:20.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:20.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:20.244 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:20.245 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:20.246 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:20.245+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:20.247 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-11T09:29:20.314 INFO:teuthology.orchestra.run.smithi107.stdout:2151 trunc from 0xcefe200 to 0xb8a5000 2024-09-11T09:29:20.333 INFO:teuthology.orchestra.run.smithi107.stdout:2152 punch from 0xa4f4c00 to 0xa4f9e00, (0x5200 bytes) 2024-09-11T09:29:20.336 INFO:teuthology.orchestra.run.smithi107.stdout:2153 write 0x5b6000 thru 0x5c5245 (0xf246 bytes) 2024-09-11T09:29:20.345 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:20.346+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:20.358 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:20.359 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:20.359 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:20.363 INFO:teuthology.orchestra.run.smithi107.stdout:2154 write 0xda4d600 thru 0xda5975a (0xc15b bytes) 2024-09-11T09:29:20.364 INFO:teuthology.orchestra.run.smithi107.stdout:2155 write 0xbd75800 thru 0xbd79edd (0x46de bytes) 2024-09-11T09:29:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:2157 read 0x52a1a00 thru 0x52a7877 (0x5e78 bytes) 2024-09-11T09:29:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:2158 read 0x25f9600 thru 0x2603f7c (0xa97d bytes) 2024-09-11T09:29:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:2159 write 0x6c75c00 thru 0x6c80cea (0xb0eb bytes) 2024-09-11T09:29:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:2160 write 0xc18f600 thru 0xc19d876 (0xe277 bytes) 2024-09-11T09:29:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:2161 read 0xcbb7c00 thru 0xcbbcb77 (0x4f78 bytes) 2024-09-11T09:29:20.365 INFO:teuthology.orchestra.run.smithi107.stdout:2162 punch from 0x47ade00 to 0x47b5200, (0x7400 bytes) 2024-09-11T09:29:20.369 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:2163 punch from 0xc565200 to 0xc575000, (0xfe00 bytes) 2024-09-11T09:29:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:2165 read 0x6519c00 thru 0x651e1ea (0x45eb bytes) 2024-09-11T09:29:20.370 INFO:teuthology.orchestra.run.smithi107.stdout:2166 write 0xc240800 thru 0xc250589 (0xfd8a bytes) 2024-09-11T09:29:20.385 INFO:teuthology.orchestra.run.smithi107.stdout:2167 trunc from 0xda59800 to 0xaa0200 2024-09-11T09:29:20.454 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:20.455+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:2172 write 0xe6f4c00 thru 0xe70160f (0xca10 bytes) 2024-09-11T09:29:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:2173 write 0x2eb3000 thru 0x2ec22a6 (0xf2a7 bytes) 2024-09-11T09:29:20.464 INFO:teuthology.orchestra.run.smithi107.stdout:2174 punch from 0xaa95e00 to 0xaa97000, (0x1200 bytes) 2024-09-11T09:29:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:2175 read 0xda75a00 thru 0xda770d3 (0x16d4 bytes) 2024-09-11T09:29:20.469 INFO:teuthology.orchestra.run.smithi107.stdout:2180 write 0x5248000 thru 0x5250063 (0x8064 bytes) 2024-09-11T09:29:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:2181 write 0xdeeb800 thru 0xdef3984 (0x8185 bytes) 2024-09-11T09:29:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:2184 write 0xa6eaa00 thru 0xa6f7678 (0xcc79 bytes) 2024-09-11T09:29:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:2191 write 0x8d7a800 thru 0x8d7d24e (0x2a4f bytes) 2024-09-11T09:29:20.477 INFO:teuthology.orchestra.run.smithi107.stdout:2192 trunc from 0xe701800 to 0x78b1600 2024-09-11T09:29:20.535 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:20.535 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:20.536 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:20.536 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:2193 write 0x9ec8a00 thru 0x9ed18ca (0x8ecb bytes) 2024-09-11T09:29:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:2197 punch from 0x31ae800 to 0x31af000, (0x800 bytes) 2024-09-11T09:29:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:2201 read 0x700e400 thru 0x7010108 (0x1d09 bytes) 2024-09-11T09:29:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:2203 read 0x6d85200 thru 0x6d8c2b3 (0x70b4 bytes) 2024-09-11T09:29:20.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:20.563 INFO:teuthology.orchestra.run.smithi107.stdout:2205 write 0xb70ec00 thru 0xb713383 (0x4784 bytes) 2024-09-11T09:29:20.563 INFO:teuthology.orchestra.run.smithi107.stdout:2206 write 0x95c8200 thru 0x95ceeb5 (0x6cb6 bytes) 2024-09-11T09:29:20.577 INFO:teuthology.orchestra.run.smithi107.stdout:2207 trunc from 0xb713400 to 0xffc00 2024-09-11T09:29:20.587 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:20.587+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:20.648 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:20.649+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:20.669 INFO:teuthology.orchestra.run.smithi107.stdout:2209 punch from 0x64000 to 0x6ba00, (0x7a00 bytes) 2024-09-11T09:29:20.691 INFO:teuthology.orchestra.run.smithi107.stdout:2211 write 0x8272800 thru 0x827a432 (0x7c33 bytes) 2024-09-11T09:29:20.699 INFO:teuthology.orchestra.run.smithi107.stdout:2213 trunc from 0x827a600 to 0xe62ac00 2024-09-11T09:29:20.714 INFO:teuthology.orchestra.run.smithi107.stdout:2216 write 0x7f15800 thru 0x7f1a871 (0x5072 bytes) 2024-09-11T09:29:20.715 INFO:teuthology.orchestra.run.smithi107.stdout:2219 read 0x132a200 thru 0x132a310 (0x111 bytes) 2024-09-11T09:29:20.717 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.279833750000002, 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5081 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7555585560000004, 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043672824999999998, 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5081 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961281+0000", 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961285+0000", 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3560000000000001e-06 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961287+0000", 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5409999999999999e-06 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004781+0000", 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043493472999999998 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004856+0000", 2024-09-11T09:29:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5741000000000005e-05 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004947+0000", 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1222000000000002e-05 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5083 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7555429819999997, 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043785219, 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5083 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961291+0000", 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961292+0000", 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961295+0000", 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0149999999999999e-06 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961297+0000", 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004977+0000", 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043679791000000003 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005021+0000", 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4295000000000002e-05 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005075+0000", 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999998e-05 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5088 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 905216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7555334350000003, 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043963918999999997, 2024-09-11T09:29:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5088 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961301+0000", 2024-09-11T09:29:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.13e-06 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961305+0000", 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4529999999999998e-06 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961309+0000", 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005096+0000", 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043787848999999997 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005175+0000", 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8487999999999998e-05 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005264+0000", 2024-09-11T09:29:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8786e-05 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5097 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1605632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7554965879999997, 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044049360000000003, 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5097 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961339+0000", 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1440000000000001e-06 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961345+0000", 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7470000000000002e-06 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961350+0000", 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209999999999996e-06 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005291+0000", 2024-09-11T09:29:20.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043940405000000002 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005332+0000", 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1542999999999998e-05 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005386+0000", 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999998e-05 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5100 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7554767370000004, 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044229475999999997, 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5100 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961357+0000", 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961359+0000", 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.052e-06 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961365+0000", 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:29:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961369+0000", 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6689999999999997e-06 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005422+0000", 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044052701 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005499+0000", 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6664999999999999e-05 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005586+0000", 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7230000000000003e-05 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5105 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7554581559999995, 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044370891000000003, 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5105 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2024-09-11T09:29:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961377+0000", 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9120000000000001e-06 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961383+0000", 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961388+0000", 2024-09-11T09:29:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005629+0000", 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044240743999999999 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005687+0000", 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7951000000000001e-05 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005746+0000", 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9444999999999998e-05 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5108 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1607680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7554397989999995, 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581704, 2024-09-11T09:29:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5108 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961394+0000", 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961396+0000", 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961401+0000", 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8119999999999996e-06 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961406+0000", 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4920000000000004e-06 2024-09-11T09:29:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005789+0000", 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383498 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005882+0000", 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2541999999999996e-05 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005975+0000", 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3110999999999997e-05 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5111 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7554217550000004, 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044694581999999997, 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5111 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961412+0000", 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961414+0000", 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.996e-06 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961420+0000", 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8000000000000004e-06 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961425+0000", 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5149999999999997e-06 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006007+0000", 2024-09-11T09:29:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581786999999998 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006051+0000", 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3998999999999997e-05 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006106+0000", 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5145999999999997e-05 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5116 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1632256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7554023510000007, 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044886862999999999, 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5116 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961433+0000", 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961439+0000", 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2024-09-11T09:29:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961444+0000", 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809999999999998e-06 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006144+0000", 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044699782 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006221+0000", 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7270999999999999e-05 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006318+0000", 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6914000000000001e-05 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5121 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1611776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7553839300000007, 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045003543, 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5121 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961450+0000", 2024-09-11T09:29:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961452+0000", 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0030000000000001e-06 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961459+0000", 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961464+0000", 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006348+0000", 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044884563000000002 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006396+0000", 2024-09-11T09:29:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7817e-05 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006453+0000", 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6762000000000003e-05 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5124 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.755363461, 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045118734000000001, 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5124 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961472+0000", 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929999999999999e-06 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961478+0000", 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.592e-06 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961482+0000", 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006470+0000", 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044987680000000002 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006529+0000", 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9116000000000001e-05 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006589+0000", 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9463000000000003e-05 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5129 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1624064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7553450920000007, 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045244390000000002, 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5129 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000002e-07 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961490+0000", 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961496+0000", 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961500+0000", 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3819999999999997e-06 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006606+0000", 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045106159 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006669+0000", 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2188999999999997e-05 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006733+0000", 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3848000000000003e-05 2024-09-11T09:29:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5134 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7553101210000008, 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045344917999999998, 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5134 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961524+0000", 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961531+0000", 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961541+0000", 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0059e-05 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961546+0000", 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006750+0000", 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045204167000000003 2024-09-11T09:29:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006808+0000", 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8229000000000001e-05 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006868+0000", 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9953e-05 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5135 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1622016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7552804200000001, 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045554071000000002, 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5135 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7100000000000002e-07 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961555+0000", 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1679999999999998e-06 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961561+0000", 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961566+0000", 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4170000000000002e-06 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006908+0000", 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045342423999999999 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007004+0000", 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5307000000000001e-05 2024-09-11T09:29:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007107+0000", 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103253 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5138 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1634304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7552606510000004, 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045679006000000001, 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5138 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961805+0000", 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023282700000000001 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961816+0000", 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961828+0000", 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2318e-05 2024-09-11T09:29:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961835+0000", 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007139+0000", 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045303738000000003 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007188+0000", 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9014999999999997e-05 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007252+0000", 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3921e-05 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5143 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1613824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7549872670000006, 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045634479999999998, 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5143 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961994+0000", 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147949 2024-09-11T09:29:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961997+0000", 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5580000000000001e-06 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962007+0000", 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6579999999999997e-06 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962014+0000", 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007290+0000", 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045276416 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007380+0000", 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9686999999999997e-05 2024-09-11T09:29:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007480+0000", 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100762 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5148 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1626112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7548093770000008, 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045674845999999998, 2024-09-11T09:29:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5148 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962412+0000", 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038829000000000001 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962417+0000", 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962427+0000", 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962434+0000", 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7379999999999997e-06 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007514+0000", 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045080264000000002 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007600+0000", 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5970000000000005e-05 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007699+0000", 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8706e-05 2024-09-11T09:29:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5153 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1615872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7543879659999995, 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045448919999999997, 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5153 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962446+0000", 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962448+0000", 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962455+0000", 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962458+0000", 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007712+0000", 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045253466999999999 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007784+0000", 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1939000000000004e-05 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007894+0000", 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5156 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.754368908, 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045635835999999999, 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5156 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962886+0000", 2024-09-11T09:29:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042155799999999999 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962890+0000", 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962899+0000", 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0440000000000001e-06 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962906+0000", 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1169999999999998e-06 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007917+0000", 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045011412000000001 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007994+0000", 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6650999999999995e-05 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.008100+0000", 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106164 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:20.816 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T09:29:20.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:20.819 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:20.818+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2223 write 0x4453800 thru 0x445e809 (0xb00a bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2229 trunc from 0xe62ac00 to 0xe79a600 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2231 read 0x5185e00 thru 0x5190f23 (0xb124 bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2233 read 0xb3d6a00 thru 0xb3daa7e (0x407f bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2236 read 0x547800 thru 0x55175f (0x9f60 bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2238 trunc from 0xe79a600 to 0xe9c9800 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2239 write 0xb380400 thru 0xb381ad3 (0x16d4 bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2240 trunc from 0xe9c9800 to 0x5db8400 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2243 read 0x5b7cc00 thru 0x5b89fd8 (0xd3d9 bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2244 punch from 0x37c4000 to 0x37c7c00, (0x3c00 bytes) 2024-09-11T09:29:20.820 INFO:teuthology.orchestra.run.smithi107.stdout:2245 trunc from 0x5db8400 to 0x674000 2024-09-11T09:29:20.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T09:29:20.827 INFO:teuthology.orchestra.run.smithi107.stdout:2247 write 0xcce8800 thru 0xccf5cf1 (0xd4f2 bytes) 2024-09-11T09:29:20.827 INFO:teuthology.orchestra.run.smithi107.stdout:2249 read 0x743ac00 thru 0x7440763 (0x5b64 bytes) 2024-09-11T09:29:20.845 INFO:teuthology.orchestra.run.smithi107.stdout:2252 write 0xee29c00 thru 0xee334c5 (0x98c6 bytes) 2024-09-11T09:29:20.846 INFO:teuthology.orchestra.run.smithi107.stdout:2254 write 0x4fc6000 thru 0x4fc6e92 (0xe93 bytes) 2024-09-11T09:29:20.846 INFO:teuthology.orchestra.run.smithi107.stdout:2257 punch from 0x450cc00 to 0x4512e00, (0x6200 bytes) 2024-09-11T09:29:20.848 INFO:teuthology.orchestra.run.smithi107.stdout:2261 punch from 0x535d400 to 0x536d200, (0xfe00 bytes) 2024-09-11T09:29:20.848 INFO:teuthology.orchestra.run.smithi107.stdout:2262 read 0x15a5800 thru 0x15b38c2 (0xe0c3 bytes) 2024-09-11T09:29:20.848 INFO:teuthology.orchestra.run.smithi107.stdout:2263 write 0x1071600 thru 0x1079bb0 (0x85b1 bytes) 2024-09-11T09:29:20.848 INFO:teuthology.orchestra.run.smithi107.stdout:2266 write 0x8cdd200 thru 0x8ce2309 (0x510a bytes) 2024-09-11T09:29:20.865 INFO:teuthology.orchestra.run.smithi107.stdout:2267 trunc from 0xee33600 to 0x203a00 2024-09-11T09:29:20.918 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:20.918+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2269 write 0x656fa00 thru 0x657e30f (0xe910 bytes) 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2271 read 0x162bc00 thru 0x163a3d6 (0xe7d7 bytes) 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2272 write 0x586a400 thru 0x586ed63 (0x4964 bytes) 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2273 read 0x678400 thru 0x6818f0 (0x94f1 bytes) 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2275 read 0x10f1600 thru 0x10fc49a (0xae9b bytes) 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2276 read 0x4db4e00 thru 0x4dbe1d7 (0x93d8 bytes) 2024-09-11T09:29:20.943 INFO:teuthology.orchestra.run.smithi107.stdout:2278 trunc from 0x657e400 to 0x68b4e00 2024-09-11T09:29:20.967 INFO:teuthology.orchestra.run.smithi107.stdout:2279 write 0x607f000 thru 0x6081ffd (0x2ffe bytes) 2024-09-11T09:29:20.977 INFO:teuthology.orchestra.run.smithi107.stdout:2280 trunc from 0x68b4e00 to 0x965a000 2024-09-11T09:29:20.984 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:20.984 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:20.984 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:20.994 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:20.996 INFO:teuthology.orchestra.run.smithi107.stdout:2282 punch from 0xeac400 to 0xeba800, (0xe400 bytes) 2024-09-11T09:29:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:2286 write 0x885dc00 thru 0x8865d01 (0x8102 bytes) 2024-09-11T09:29:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:2288 read 0x8fbf000 thru 0x8fc0a86 (0x1a87 bytes) 2024-09-11T09:29:20.998 INFO:teuthology.orchestra.run.smithi107.stdout:2292 read 0x7b97a00 thru 0x7b9e454 (0x6a55 bytes) 2024-09-11T09:29:21.020 INFO:teuthology.orchestra.run.smithi107.stdout:2295 write 0xe5f9e00 thru 0xe606de6 (0xcfe7 bytes) 2024-09-11T09:29:21.021 INFO:teuthology.orchestra.run.smithi107.stdout:2296 write 0x27d2c00 thru 0x27d50c1 (0x24c2 bytes) 2024-09-11T09:29:21.032 INFO:teuthology.orchestra.run.smithi107.stdout:2297 trunc from 0xe606e00 to 0x4fe7400 2024-09-11T09:29:21.038 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:21.038+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:21.071 INFO:teuthology.orchestra.run.smithi107.stdout:2298 punch from 0x3064000 to 0x306f600, (0xb600 bytes) 2024-09-11T09:29:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:2299 read 0x4fa8e00 thru 0x4facdd2 (0x3fd3 bytes) 2024-09-11T09:29:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:2300 punch from 0x4650800 to 0x4654a00, (0x4200 bytes) 2024-09-11T09:29:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:2302 read 0x4fc8e00 thru 0x4fd3da5 (0xafa6 bytes) 2024-09-11T09:29:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:2304 punch from 0x4e57000 to 0x4e5b400, (0x4400 bytes) 2024-09-11T09:29:21.074 INFO:teuthology.orchestra.run.smithi107.stdout:2305 read 0x20eac00 thru 0x20f56f1 (0xaaf2 bytes) 2024-09-11T09:29:21.103 INFO:teuthology.orchestra.run.smithi107.stdout:2306 write 0x8190c00 thru 0x81a0244 (0xf645 bytes) 2024-09-11T09:29:21.104 INFO:teuthology.orchestra.run.smithi107.stdout:2307 punch from 0x588000 to 0x592a00, (0xaa00 bytes) 2024-09-11T09:29:21.107 INFO:teuthology.orchestra.run.smithi107.stdout:2309 write 0x3721c00 thru 0x372bc64 (0xa065 bytes) 2024-09-11T09:29:21.107 INFO:teuthology.orchestra.run.smithi107.stdout:2310 read 0x209600 thru 0x2174d7 (0xded8 bytes) 2024-09-11T09:29:21.107 INFO:teuthology.orchestra.run.smithi107.stdout:2311 punch from 0x632e400 to 0x6335000, (0x6c00 bytes) 2024-09-11T09:29:21.108 INFO:teuthology.orchestra.run.smithi107.stdout:2313 punch from 0x6b2c000 to 0x6b2c800, (0x800 bytes) 2024-09-11T09:29:21.108 INFO:teuthology.orchestra.run.smithi107.stdout:2314 punch from 0x242a800 to 0x2432800, (0x8000 bytes) 2024-09-11T09:29:21.108 INFO:teuthology.orchestra.run.smithi107.stdout:2316 read 0x4cde000 thru 0x4ce107c (0x307d bytes) 2024-09-11T09:29:21.108 INFO:teuthology.orchestra.run.smithi107.stdout:2318 write 0x32ac200 thru 0x32b252b (0x632c bytes) 2024-09-11T09:29:21.108 INFO:teuthology.orchestra.run.smithi107.stdout:2319 read 0x252b200 thru 0x252ee22 (0x3c23 bytes) 2024-09-11T09:29:21.119 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:21.120+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:21.120 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:21.120 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:21.121 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:21.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:21.147 INFO:teuthology.orchestra.run.smithi107.stdout:2320 write 0xbb47000 thru 0xbb547f6 (0xd7f7 bytes) 2024-09-11T09:29:21.147 INFO:teuthology.orchestra.run.smithi107.stdout:2321 punch from 0xacc7600 to 0xacd3c00, (0xc600 bytes) 2024-09-11T09:29:21.167 INFO:teuthology.orchestra.run.smithi107.stdout:2323 write 0xd1b4400 thru 0xd1be31d (0x9f1e bytes) 2024-09-11T09:29:21.168 INFO:teuthology.orchestra.run.smithi107.stdout:2324 write 0xc455800 thru 0xc46182e (0xc02f bytes) 2024-09-11T09:29:21.169 INFO:teuthology.orchestra.run.smithi107.stdout:2327 punch from 0x346f600 to 0x347aa00, (0xb400 bytes) 2024-09-11T09:29:21.170 INFO:teuthology.orchestra.run.smithi107.stdout:2328 read 0x2744a00 thru 0x274e07c (0x967d bytes) 2024-09-11T09:29:21.170 INFO:teuthology.orchestra.run.smithi107.stdout:2330 write 0x4c27400 thru 0x4c31dc3 (0xa9c4 bytes) 2024-09-11T09:29:21.170 INFO:teuthology.orchestra.run.smithi107.stdout:2331 punch from 0xfd200 to 0x100c00, (0x3a00 bytes) 2024-09-11T09:29:21.171 INFO:teuthology.orchestra.run.smithi107.stdout:2333 read 0x54fe00 thru 0x552b99 (0x2d9a bytes) 2024-09-11T09:29:21.206 INFO:teuthology.orchestra.run.smithi107.stdout:2334 write 0xe13fe00 thru 0xe144a66 (0x4c67 bytes) 2024-09-11T09:29:21.206 INFO:teuthology.orchestra.run.smithi107.stdout:2335 read 0x285e00 thru 0x294ccd (0xeece bytes) 2024-09-11T09:29:21.207 INFO:teuthology.orchestra.run.smithi107.stdout:2336 punch from 0x2513c00 to 0x2520e00, (0xd200 bytes) 2024-09-11T09:29:21.216 INFO:teuthology.orchestra.run.smithi107.stdout:2337 trunc from 0xe144c00 to 0x71d4000 2024-09-11T09:29:21.220 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:21.220+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:21.257 INFO:teuthology.orchestra.run.smithi107.stdout:2338 write 0x97a3c00 thru 0x97aaf84 (0x7385 bytes) 2024-09-11T09:29:21.257 INFO:teuthology.orchestra.run.smithi107.stdout:2339 read 0x1993a00 thru 0x199552d (0x1b2e bytes) 2024-09-11T09:29:21.257 INFO:teuthology.orchestra.run.smithi107.stdout:2342 write 0x676aa00 thru 0x6770c22 (0x6223 bytes) 2024-09-11T09:29:21.281 INFO:teuthology.orchestra.run.smithi107.stdout:2343 write 0xbcfec00 thru 0xbd08411 (0x9812 bytes) 2024-09-11T09:29:21.284 INFO:teuthology.orchestra.run.smithi107.stdout:2344 trunc from 0xbd08600 to 0x98a8200 2024-09-11T09:29:21.299 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:21.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931374704, 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:21.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:21.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:21.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931337319000001, 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:21.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:21.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931318001000001, 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931306755000001, 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:21.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.319 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931264850000002, 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:21.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931247446, 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:21.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931229551000001, 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:21.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931209371000001, 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:21.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931188386999999, 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:21.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931161289999999, 2024-09-11T09:29:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.348 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.931142003000001, 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:21.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:21.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.930937643, 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:21.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:21.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.930403887000001, 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:21.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.930354993999998, 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:21.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.930306905999998, 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.368 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:21.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.930268867999999, 2024-09-11T09:29:21.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:21.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.930045025999998, 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:21.377 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:21.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:21.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.929867924, 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:21.382 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:21.386 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.929722369, 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.387 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:21.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 17.929600983, 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:21.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:21.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.396 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:21.398 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:21.399 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:21.397+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:2346 read 0x3f2fe00 thru 0x3f353eb (0x55ec bytes) 2024-09-11T09:29:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:2347 write 0x7628800 thru 0x762ca4b (0x424c bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2348 read 0x1030400 thru 0x103d311 (0xcf12 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2352 read 0x81bbc00 thru 0x81c78e0 (0xbce1 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2354 write 0xd63ca00 thru 0xd646e97 (0xa498 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2355 write 0x752400 thru 0x75a421 (0x8022 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2357 read 0xc9800 thru 0xd12f0 (0x7af1 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2358 read 0x3e38400 thru 0x3e3dff9 (0x5bfa bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2360 trunc from 0xd647000 to 0x4ca2800 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2361 read 0x9dfe00 thru 0x9edc70 (0xde71 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2362 write 0x9cec200 thru 0x9cf2fd8 (0x6dd9 bytes) 2024-09-11T09:29:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:2363 trunc from 0x9cf3000 to 0xc9e4400 2024-09-11T09:29:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:2364 read 0x9991600 thru 0x999b090 (0x9a91 bytes) 2024-09-11T09:29:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:2365 read 0x8a26800 thru 0x8a2f81f (0x9020 bytes) 2024-09-11T09:29:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:2368 write 0x3154e00 thru 0x3163302 (0xe503 bytes) 2024-09-11T09:29:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:2369 write 0xe4c0800 thru 0xe4cc9c3 (0xc1c4 bytes) 2024-09-11T09:29:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:2370 punch from 0x602ec00 to 0x6038000, (0x9400 bytes) 2024-09-11T09:29:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:2371 trunc from 0xe4cca00 to 0x4cb7000 2024-09-11T09:29:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:2373 punch from 0x407fc00 to 0x4088c00, (0x9000 bytes) 2024-09-11T09:29:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:2376 write 0x2748200 thru 0x2750461 (0x8262 bytes) 2024-09-11T09:29:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:2381 read 0x4670000 thru 0x4679650 (0x9651 bytes) 2024-09-11T09:29:21.498 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:21.499+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:2383 write 0x8963a00 thru 0x8969a08 (0x6009 bytes) 2024-09-11T09:29:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:2385 punch from 0xda7800 to 0xdad800, (0x6000 bytes) 2024-09-11T09:29:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:2388 read 0x60a4e00 thru 0x60a75fb (0x27fc bytes) 2024-09-11T09:29:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:2390 read 0x49fc600 thru 0x49ff53a (0x2f3b bytes) 2024-09-11T09:29:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:2391 punch from 0x3407c00 to 0x340ce00, (0x5200 bytes) 2024-09-11T09:29:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:2392 read 0x29bb800 thru 0x29c8b3b (0xd33c bytes) 2024-09-11T09:29:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:2393 read 0x116ee00 thru 0x1172a75 (0x3c76 bytes) 2024-09-11T09:29:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:2396 trunc from 0x8969c00 to 0x954d600 2024-09-11T09:29:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:2399 write 0x1180c00 thru 0x118fc50 (0xf051 bytes) 2024-09-11T09:29:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:2400 read 0x71f200 thru 0x721179 (0x1f7a bytes) 2024-09-11T09:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:2401 read 0x5ae0000 thru 0x5ae8a5b (0x8a5c bytes) 2024-09-11T09:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:2403 write 0x2282200 thru 0x2284881 (0x2682 bytes) 2024-09-11T09:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:2404 read 0x25ba400 thru 0x25c95a5 (0xf1a6 bytes) 2024-09-11T09:29:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:2406 punch from 0x2a5a000 to 0x2a69200, (0xf200 bytes) 2024-09-11T09:29:21.561 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:21.561 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:21.562 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:2407 write 0xb7db000 thru 0xb7e50d1 (0xa0d2 bytes) 2024-09-11T09:29:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:2409 read 0x89ed600 thru 0x89f20e9 (0x4aea bytes) 2024-09-11T09:29:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:2412 write 0x5d30200 thru 0x5d34f9b (0x4d9c bytes) 2024-09-11T09:29:21.572 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:21.590 INFO:teuthology.orchestra.run.smithi107.stdout:2414 write 0xc511600 thru 0xc51a6d4 (0x90d5 bytes) 2024-09-11T09:29:21.592 INFO:teuthology.orchestra.run.smithi107.stdout:2415 write 0x14e600 thru 0x1527eb (0x41ec bytes) 2024-09-11T09:29:21.592 INFO:teuthology.orchestra.run.smithi107.stdout:2417 write 0x3a20200 thru 0x3a26e8b (0x6c8c bytes) 2024-09-11T09:29:21.592 INFO:teuthology.orchestra.run.smithi107.stdout:2420 write 0x8ba800 thru 0x8c753c (0xcd3d bytes) 2024-09-11T09:29:21.592 INFO:teuthology.orchestra.run.smithi107.stdout:2421 read 0x724f400 thru 0x725688d (0x748e bytes) 2024-09-11T09:29:21.592 INFO:teuthology.orchestra.run.smithi107.stdout:2422 read 0x7799200 thru 0x77a426a (0xb06b bytes) 2024-09-11T09:29:21.613 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:21.613+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:21.631 INFO:teuthology.orchestra.run.smithi107.stdout:2424 write 0xd86f400 thru 0xd8725ca (0x31cb bytes) 2024-09-11T09:29:21.632 INFO:teuthology.orchestra.run.smithi107.stdout:2427 punch from 0x5751600 to 0x5756000, (0x4a00 bytes) 2024-09-11T09:29:21.634 INFO:teuthology.orchestra.run.smithi107.stdout:2431 read 0xcfe5e00 thru 0xcfeedc6 (0x8fc7 bytes) 2024-09-11T09:29:21.634 INFO:teuthology.orchestra.run.smithi107.stdout:2433 read 0x6a3b200 thru 0x6a3ba13 (0x814 bytes) 2024-09-11T09:29:21.640 INFO:teuthology.orchestra.run.smithi107.stdout:2435 trunc from 0xd872600 to 0x81dda00 2024-09-11T09:29:21.671 INFO:teuthology.orchestra.run.smithi107.stdout:2436 write 0x5875a00 thru 0x58840f4 (0xe6f5 bytes) 2024-09-11T09:29:21.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:21.672 INFO:teuthology.orchestra.run.smithi107.stdout:2437 punch from 0x6720400 to 0x6720a00, (0x600 bytes) 2024-09-11T09:29:21.674 INFO:teuthology.orchestra.run.smithi107.stdout:2439 write 0x4135600 thru 0x413bc0a (0x660b bytes) 2024-09-11T09:29:21.674 INFO:teuthology.orchestra.run.smithi107.stdout:2441 flatten 2024-09-11T09:29:21.695 INFO:teuthology.orchestra.run.smithi107.stdout:2442 read 0x1473a00 thru 0x147b015 (0x7616 bytes) 2024-09-11T09:29:21.695 INFO:teuthology.orchestra.run.smithi107.stdout:2443 write 0x5120000 thru 0x5123c15 (0x3c16 bytes) 2024-09-11T09:29:21.695 INFO:teuthology.orchestra.run.smithi107.stdout:2445 write 0x5d3400 thru 0x5da62d (0x722e bytes) 2024-09-11T09:29:21.714 INFO:teuthology.orchestra.run.smithi107.stdout:2448 write 0xaa5ca00 thru 0xaa67431 (0xaa32 bytes) 2024-09-11T09:29:21.715 INFO:teuthology.orchestra.run.smithi107.stdout:2450 write 0x66fc200 thru 0x6704dc6 (0x8bc7 bytes) 2024-09-11T09:29:21.715 INFO:teuthology.orchestra.run.smithi107.stdout:2454 write 0x1703000 thru 0x170b9c2 (0x89c3 bytes) 2024-09-11T09:29:21.715 INFO:teuthology.orchestra.run.smithi107.stdout:2455 write 0x24e2600 thru 0x24f0d61 (0xe762 bytes) 2024-09-11T09:29:21.716 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:21.717+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:21.721 INFO:teuthology.orchestra.run.smithi107.stdout:2456 trunc from 0xaa67600 to 0xca87c00 2024-09-11T09:29:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:21.739 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:21.739 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:21.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:21.749 INFO:teuthology.orchestra.run.smithi107.stdout:2458 trunc from 0xca87c00 to 0xe0f2e00 2024-09-11T09:29:21.762 INFO:teuthology.orchestra.run.smithi107.stdout:2459 read 0xb7a2a00 thru 0xb7a3197 (0x798 bytes) 2024-09-11T09:29:21.762 INFO:teuthology.orchestra.run.smithi107.stdout:2461 write 0x1230600 thru 0x123e8ad (0xe2ae bytes) 2024-09-11T09:29:21.775 INFO:teuthology.orchestra.run.smithi107.stdout:2463 trunc from 0xe0f2e00 to 0x3e03c00 2024-09-11T09:29:21.801 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:21.801+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:21.805 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T09:29:21.812 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:21.812 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:21.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T09:29:21.827 INFO:teuthology.orchestra.run.smithi107.stdout:2464 read 0x38a7c00 thru 0x38a80c7 (0x4c8 bytes) 2024-09-11T09:29:21.829 INFO:teuthology.orchestra.run.smithi107.stdout:2466 read 0x1410600 thru 0x141c63d (0xc03e bytes) 2024-09-11T09:29:21.829 INFO:teuthology.orchestra.run.smithi107.stdout:2467 read 0x318800 thru 0x3239eb (0xb1ec bytes) 2024-09-11T09:29:21.829 INFO:teuthology.orchestra.run.smithi107.stdout:2470 read 0x3290e00 thru 0x3296293 (0x5494 bytes) 2024-09-11T09:29:21.831 INFO:teuthology.orchestra.run.smithi107.stdout:2472 punch from 0x5b4800 to 0x5bec00, (0xa400 bytes) 2024-09-11T09:29:21.831 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:21.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:29:21.850 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:21.850 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:21.851 INFO:teuthology.orchestra.run.smithi107.stdout:2474 write 0xc76b400 thru 0xc7740e4 (0x8ce5 bytes) 2024-09-11T09:29:21.863 INFO:teuthology.orchestra.run.smithi107.stdout:2476 trunc from 0xc774200 to 0x2282c00 2024-09-11T09:29:21.870 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:21.871 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4835 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.88803482, 2024-09-11T09:29:21.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033722624999999999, 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4835 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:21.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7800000000000002e-07 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983045+0000", 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.657e-06 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983049+0000", 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3420000000000001e-06 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983052+0000", 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.74e-06 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016669+0000", 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033617247000000003 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016711+0000", 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1149999999999997e-05 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016766+0000", 2024-09-11T09:29:21.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5111000000000001e-05 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.887978971000001, 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.877 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:21.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.887957196, 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.882 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:21.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.887911152000001, 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:21.886 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:21.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.890 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.887899482, 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:21.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.887026072999999, 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:21.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:21.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886662598000001, 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.900 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886611213, 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:21.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886575512, 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:21.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:21.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886553474999999, 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:21.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.88651789, 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:21.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:21.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886457546999999, 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:21.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886408863, 2024-09-11T09:29:21.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:21.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886352399, 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:21.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.937 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886332844, 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:21.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:21.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886285081, 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:21.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886171343000001, 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:21.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:21.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:21.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886152599000001, 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.953 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:21.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.957 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886113610000001, 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.958 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:21.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 10.886090503, 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:21.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:21.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:21.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:21.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:21.968 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:21.969 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:21.970 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:2477 write 0xb40ce00 thru 0xb40ece3 (0x1ee4 bytes) 2024-09-11T09:29:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:2481 read 0x379c00 thru 0x38465b (0xaa5c bytes) 2024-09-11T09:29:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:2482 read 0x1960000 thru 0x1963dab (0x3dac bytes) 2024-09-11T09:29:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:2483 trunc from 0xb40ee00 to 0x7ba5000 2024-09-11T09:29:21.971 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:21.971 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:21.969+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:21.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:21.972 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:2485 punch from 0x580f800 to 0x581ae00, (0xb600 bytes) 2024-09-11T09:29:21.995 INFO:teuthology.orchestra.run.smithi107.stdout:2490 write 0xb172e00 thru 0xb1774f7 (0x46f8 bytes) 2024-09-11T09:29:21.996 INFO:teuthology.orchestra.run.smithi107.stdout:2491 write 0x59cc600 thru 0x59cfb41 (0x3542 bytes) 2024-09-11T09:29:21.996 INFO:teuthology.orchestra.run.smithi107.stdout:2492 write 0x4549000 thru 0x4558e70 (0xfe71 bytes) 2024-09-11T09:29:21.996 INFO:teuthology.orchestra.run.smithi107.stdout:2495 punch from 0x2c11000 to 0x2c1d200, (0xc200 bytes) 2024-09-11T09:29:22.009 INFO:teuthology.orchestra.run.smithi107.stdout:2496 trunc from 0xb177600 to 0x1405400 2024-09-11T09:29:22.056 INFO:teuthology.orchestra.run.smithi107.stdout:2497 read 0x85a800 thru 0x85bfc3 (0x17c4 bytes) 2024-09-11T09:29:22.057 INFO:teuthology.orchestra.run.smithi107.stdout:2498 read 0x28c800 thru 0x29b943 (0xf144 bytes) 2024-09-11T09:29:22.070 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:22.070+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:22.075 INFO:teuthology.orchestra.run.smithi107.stdout:2502 write 0x34fce00 thru 0x350786d (0xaa6e bytes) 2024-09-11T09:29:22.075 INFO:teuthology.orchestra.run.smithi107.stdout:2503 write 0x2e91800 thru 0x2e93d37 (0x2538 bytes) 2024-09-11T09:29:22.075 INFO:teuthology.orchestra.run.smithi107.stdout:2504 read 0x2c0800 thru 0x2cca58 (0xc259 bytes) 2024-09-11T09:29:22.080 INFO:teuthology.orchestra.run.smithi107.stdout:2506 trunc from 0x3507a00 to 0x7b66600 2024-09-11T09:29:22.108 INFO:teuthology.orchestra.run.smithi107.stdout:2508 read 0x4b200 thru 0x53596 (0x8397 bytes) 2024-09-11T09:29:22.113 INFO:teuthology.orchestra.run.smithi107.stdout:2509 trunc from 0x7b66600 to 0x3963000 2024-09-11T09:29:22.129 INFO:teuthology.orchestra.run.smithi107.stdout:2511 punch from 0x3280a00 to 0x3284a00, (0x4000 bytes) 2024-09-11T09:29:22.136 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:22.136 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:22.146 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:22.153 INFO:teuthology.orchestra.run.smithi107.stdout:2512 write 0x5ed0a00 thru 0x5edcc33 (0xc234 bytes) 2024-09-11T09:29:22.153 INFO:teuthology.orchestra.run.smithi107.stdout:2513 read 0x327f400 thru 0x32808a9 (0x14aa bytes) 2024-09-11T09:29:22.153 INFO:teuthology.orchestra.run.smithi107.stdout:2515 write 0x5930e00 thru 0x5934c92 (0x3e93 bytes) 2024-09-11T09:29:22.171 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:22.171+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:22.189 INFO:teuthology.orchestra.run.smithi107.stdout:2517 write 0xc825c00 thru 0xc8304b6 (0xa8b7 bytes) 2024-09-11T09:29:22.196 INFO:teuthology.orchestra.run.smithi107.stdout:2518 read 0x9b07400 thru 0x9b0ddea (0x69eb bytes) 2024-09-11T09:29:22.196 INFO:teuthology.orchestra.run.smithi107.stdout:2520 punch from 0xb5ff200 to 0xb60b600, (0xc400 bytes) 2024-09-11T09:29:22.198 INFO:teuthology.orchestra.run.smithi107.stdout:2522 punch from 0x5478a00 to 0x5488800, (0xfe00 bytes) 2024-09-11T09:29:22.199 INFO:teuthology.orchestra.run.smithi107.stdout:2526 punch from 0x9340400 to 0x9343000, (0x2c00 bytes) 2024-09-11T09:29:22.199 INFO:teuthology.orchestra.run.smithi107.stdout:2527 write 0xfb3000 thru 0xfbffc7 (0xcfc8 bytes) 2024-09-11T09:29:22.199 INFO:teuthology.orchestra.run.smithi107.stdout:2528 read 0x527f200 thru 0x5283315 (0x4116 bytes) 2024-09-11T09:29:22.199 INFO:teuthology.orchestra.run.smithi107.stdout:2532 write 0xc7af200 thru 0xc7baaae (0xb8af bytes) 2024-09-11T09:29:22.205 INFO:teuthology.orchestra.run.smithi107.stdout:2533 trunc from 0xc830600 to 0x71cec00 2024-09-11T09:29:22.255 INFO:teuthology.orchestra.run.smithi107.stdout:2535 trunc from 0x71cec00 to 0xb7d4200 2024-09-11T09:29:22.271 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:22.272+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:22.273 INFO:teuthology.orchestra.run.smithi107.stdout:2538 read 0x2f53e00 thru 0x2f56331 (0x2532 bytes) 2024-09-11T09:29:22.274 INFO:teuthology.orchestra.run.smithi107.stdout:2539 read 0x54c1e00 thru 0x54ce907 (0xcb08 bytes) 2024-09-11T09:29:22.275 INFO:teuthology.orchestra.run.smithi107.stdout:2541 read 0x2a39400 thru 0x2a3b9f4 (0x25f5 bytes) 2024-09-11T09:29:22.275 INFO:teuthology.orchestra.run.smithi107.stdout:2545 write 0x47aa400 thru 0x47b3405 (0x9006 bytes) 2024-09-11T09:29:22.275 INFO:teuthology.orchestra.run.smithi107.stdout:2547 write 0x281b800 thru 0x28263c3 (0xabc4 bytes) 2024-09-11T09:29:22.275 INFO:teuthology.orchestra.run.smithi107.stdout:2548 write 0x7a1b400 thru 0x7a2a6de (0xf2df bytes) 2024-09-11T09:29:22.275 INFO:teuthology.orchestra.run.smithi107.stdout:2549 punch from 0x58a0000 to 0x58ae200, (0xe200 bytes) 2024-09-11T09:29:22.276 INFO:teuthology.orchestra.run.smithi107.stdout:2552 read 0x61cbc00 thru 0x61d2642 (0x6a43 bytes) 2024-09-11T09:29:22.276 INFO:teuthology.orchestra.run.smithi107.stdout:2553 read 0xd81c00 thru 0xd856fc (0x3afd bytes) 2024-09-11T09:29:22.276 INFO:teuthology.orchestra.run.smithi107.stdout:2554 read 0xd72a00 thru 0xd76544 (0x3b45 bytes) 2024-09-11T09:29:22.286 INFO:teuthology.orchestra.run.smithi107.stdout:2555 trunc from 0xb7d4200 to 0x406b600 2024-09-11T09:29:22.312 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:22.312 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:22.322 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:22.355 INFO:teuthology.orchestra.run.smithi107.stdout:2557 write 0xb8bc800 thru 0xb8c27fd (0x5ffe bytes) 2024-09-11T09:29:22.356 INFO:teuthology.orchestra.run.smithi107.stdout:2559 read 0x13f2600 thru 0x13f443c (0x1e3d bytes) 2024-09-11T09:29:22.356 INFO:teuthology.orchestra.run.smithi107.stdout:2560 read 0x4b1a600 thru 0x4b2440d (0x9e0e bytes) 2024-09-11T09:29:22.356 INFO:teuthology.orchestra.run.smithi107.stdout:2562 write 0xa087400 thru 0xa089f47 (0x2b48 bytes) 2024-09-11T09:29:22.356 INFO:teuthology.orchestra.run.smithi107.stdout:2563 punch from 0x8e1c800 to 0x8e27400, (0xac00 bytes) 2024-09-11T09:29:22.356 INFO:teuthology.orchestra.run.smithi107.stdout:2565 read 0x7565600 thru 0x756c5c9 (0x6fca bytes) 2024-09-11T09:29:22.362 INFO:teuthology.orchestra.run.smithi107.stdout:2566 trunc from 0xb8c2800 to 0x66dd400 2024-09-11T09:29:22.373 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:22.373+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:22.434 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:22.434 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:22.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:22.474 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:22.474+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:22.550 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:22.551 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:22.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:22.596 INFO:teuthology.orchestra.run.smithi107.stdout:2567 read 0x29f6a00 thru 0x2a060a0 (0xf6a1 bytes) 2024-09-11T09:29:22.596 INFO:teuthology.orchestra.run.smithi107.stdout:2568 read 0x2aa5800 thru 0x2aabfcb (0x67cc bytes) 2024-09-11T09:29:22.602 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:22.603+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:22.608 INFO:teuthology.orchestra.run.smithi107.stdout:2570 write 0x9d07a00 thru 0x9d0d4a9 (0x5aaa bytes) 2024-09-11T09:29:22.609 INFO:teuthology.orchestra.run.smithi107.stdout:2571 read 0x8557800 thru 0x8561139 (0x993a bytes) 2024-09-11T09:29:22.609 INFO:teuthology.orchestra.run.smithi107.stdout:2576 read 0x8b66400 thru 0x8b733ab (0xcfac bytes) 2024-09-11T09:29:22.609 INFO:teuthology.orchestra.run.smithi107.stdout:2577 read 0x3aaa800 thru 0x3ab8e61 (0xe662 bytes) 2024-09-11T09:29:22.610 INFO:teuthology.orchestra.run.smithi107.stdout:2578 read 0x413c400 thru 0x4148cd7 (0xc8d8 bytes) 2024-09-11T09:29:22.610 INFO:teuthology.orchestra.run.smithi107.stdout:2580 write 0xa3fc00 thru 0xa4cb03 (0xcf04 bytes) 2024-09-11T09:29:22.610 INFO:teuthology.orchestra.run.smithi107.stdout:2584 read 0x31b0000 thru 0x31bf1fd (0xf1fe bytes) 2024-09-11T09:29:22.610 INFO:teuthology.orchestra.run.smithi107.stdout:2585 read 0x88a9600 thru 0x88a9e94 (0x895 bytes) 2024-09-11T09:29:22.613 INFO:teuthology.orchestra.run.smithi107.stdout:2586 trunc from 0x9d0d600 to 0x6b7d200 2024-09-11T09:29:22.650 INFO:teuthology.orchestra.run.smithi107.stdout:2587 punch from 0x851c00 to 0x857600, (0x5a00 bytes) 2024-09-11T09:29:22.650 INFO:teuthology.orchestra.run.smithi107.stdout:2588 read 0x353c200 thru 0x3542b4f (0x6950 bytes) 2024-09-11T09:29:22.651 INFO:teuthology.orchestra.run.smithi107.stdout:2590 read 0x62cfe00 thru 0x62d5522 (0x5723 bytes) 2024-09-11T09:29:22.651 INFO:teuthology.orchestra.run.smithi107.stdout:2591 punch from 0x4337600 to 0x4345c00, (0xe600 bytes) 2024-09-11T09:29:22.651 INFO:teuthology.orchestra.run.smithi107.stdout:2592 write 0x623ae00 thru 0x6242fab (0x81ac bytes) 2024-09-11T09:29:22.677 INFO:teuthology.orchestra.run.smithi107.stdout:2594 write 0x72c5600 thru 0x72d5320 (0xfd21 bytes) 2024-09-11T09:29:22.678 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:22.678+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:22.681 INFO:teuthology.orchestra.run.smithi107.stdout:2596 trunc from 0x72d5400 to 0xacbb800 2024-09-11T09:29:22.683 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:22.683 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:22.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:22.706 INFO:teuthology.orchestra.run.smithi107.stdout:2597 punch from 0xa226400 to 0xa231c00, (0xb800 bytes) 2024-09-11T09:29:22.706 INFO:teuthology.orchestra.run.smithi107.stdout:2600 write 0x8606e00 thru 0x8611f1f (0xb120 bytes) 2024-09-11T09:29:22.717 INFO:teuthology.orchestra.run.smithi107.stdout:2602 trunc from 0xacbb800 to 0x1d00e00 2024-09-11T09:29:22.764 INFO:teuthology.orchestra.run.smithi107.stdout:2603 read 0x189ba00 thru 0x189e4a5 (0x2aa6 bytes) 2024-09-11T09:29:22.774 INFO:teuthology.orchestra.run.smithi107.stdout:2605 trunc from 0x1d00e00 to 0xa758600 2024-09-11T09:29:22.781 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:22.781+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:22.784 INFO:teuthology.orchestra.run.smithi107.stdout:2606 punch from 0xb43400 to 0xb46400, (0x3000 bytes) 2024-09-11T09:29:22.786 INFO:teuthology.orchestra.run.smithi107.stdout:2610 read 0x38fb800 thru 0x38fd87d (0x207e bytes) 2024-09-11T09:29:22.786 INFO:teuthology.orchestra.run.smithi107.stdout:2613 read 0x154f800 thru 0x1558d04 (0x9505 bytes) 2024-09-11T09:29:22.786 INFO:teuthology.orchestra.run.smithi107.stdout:2614 read 0x9442600 thru 0x94498a6 (0x72a7 bytes) 2024-09-11T09:29:22.787 INFO:teuthology.orchestra.run.smithi107.stdout:2615 trunc from 0xa758600 to 0xb5b8000 2024-09-11T09:29:22.801 INFO:teuthology.orchestra.run.smithi107.stdout:2617 write 0x65da00 thru 0x669dc8 (0xc3c9 bytes) 2024-09-11T09:29:22.806 INFO:teuthology.orchestra.run.smithi107.stdout:2618 trunc from 0xb5b8000 to 0xe6df400 2024-09-11T09:29:22.812 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:29:22.815 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T09:29:22.819 INFO:teuthology.orchestra.run.smithi107.stdout:2620 write 0xd502800 thru 0xd506bf6 (0x43f7 bytes) 2024-09-11T09:29:22.826 INFO:teuthology.orchestra.run.smithi107.stdout:2621 read 0x3ac00 thru 0x40a01 (0x5e02 bytes) 2024-09-11T09:29:22.830 INFO:teuthology.orchestra.run.smithi107.stdout:2625 read 0x3808c00 thru 0x381207a (0x947b bytes) 2024-09-11T09:29:22.830 INFO:teuthology.orchestra.run.smithi107.stdout:2627 write 0x434d200 thru 0x4355169 (0x7f6a bytes) 2024-09-11T09:29:22.830 INFO:teuthology.orchestra.run.smithi107.stdout:2629 read 0xcb55400 thru 0xcb59442 (0x4043 bytes) 2024-09-11T09:29:22.831 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T09:29:22.832 INFO:teuthology.orchestra.run.smithi107.stdout:2630 trunc from 0xe6df400 to 0xcb31800 2024-09-11T09:29:22.855 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:22.855 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:22.857 INFO:teuthology.orchestra.run.smithi107.stdout:2631 punch from 0x7c5c800 to 0x7c60200, (0x3a00 bytes) 2024-09-11T09:29:22.861 INFO:teuthology.orchestra.run.smithi107.stdout:2632 read 0xa98c400 thru 0xa99ba9d (0xf69e bytes) 2024-09-11T09:29:22.861 INFO:teuthology.orchestra.run.smithi107.stdout:2634 punch from 0x8e23c00 to 0x8e2cc00, (0x9000 bytes) 2024-09-11T09:29:22.861 INFO:teuthology.orchestra.run.smithi107.stdout:2636 punch from 0x3817800 to 0x3818e00, (0x1600 bytes) 2024-09-11T09:29:22.861 INFO:teuthology.orchestra.run.smithi107.stdout:2638 write 0x46f6600 thru 0x47042bf (0xdcc0 bytes) 2024-09-11T09:29:22.862 INFO:teuthology.orchestra.run.smithi107.stdout:2639 write 0x7206000 thru 0x7207f41 (0x1f42 bytes) 2024-09-11T09:29:22.862 INFO:teuthology.orchestra.run.smithi107.stdout:2641 read 0x517fc00 thru 0x5181c0e (0x200f bytes) 2024-09-11T09:29:22.862 INFO:teuthology.orchestra.run.smithi107.stdout:2642 read 0xbc25e00 thru 0xbc2fa87 (0x9c88 bytes) 2024-09-11T09:29:22.862 INFO:teuthology.orchestra.run.smithi107.stdout:2643 read 0x126c800 thru 0x1273c18 (0x7419 bytes) 2024-09-11T09:29:22.862 INFO:teuthology.orchestra.run.smithi107.stdout:2644 read 0x6934a00 thru 0x693517a (0x77b bytes) 2024-09-11T09:29:22.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:22.893 INFO:teuthology.orchestra.run.smithi107.stdout:2645 write 0xe286000 thru 0xe28659c (0x59d bytes) 2024-09-11T09:29:22.894 INFO:teuthology.orchestra.run.smithi107.stdout:2647 write 0x9531600 thru 0x9531772 (0x173 bytes) 2024-09-11T09:29:22.894 INFO:teuthology.orchestra.run.smithi107.stdout:2649 write 0x461ea00 thru 0x461ea4c (0x4d bytes) 2024-09-11T09:29:22.895 INFO:teuthology.orchestra.run.smithi107.stdout:2651 write 0x74cca00 thru 0x74d16e0 (0x4ce1 bytes) 2024-09-11T09:29:22.895 INFO:teuthology.orchestra.run.smithi107.stdout:2655 read 0x56e6200 thru 0x56f1d97 (0xbb98 bytes) 2024-09-11T09:29:22.895 INFO:teuthology.orchestra.run.smithi107.stdout:2656 write 0xcd5be00 thru 0xcd68669 (0xc86a bytes) 2024-09-11T09:29:22.895 INFO:teuthology.orchestra.run.smithi107.stdout:2658 read 0xb751c00 thru 0xb75691b (0x4d1c bytes) 2024-09-11T09:29:22.895 INFO:teuthology.orchestra.run.smithi107.stdout:2659 punch from 0xafe9c00 to 0xafefc00, (0x6000 bytes) 2024-09-11T09:29:22.896 INFO:teuthology.orchestra.run.smithi107.stdout:2660 read 0xb02f000 thru 0xb033387 (0x4388 bytes) 2024-09-11T09:29:22.896 INFO:teuthology.orchestra.run.smithi107.stdout:2662 read 0x1097c00 thru 0x10a7463 (0xf864 bytes) 2024-09-11T09:29:22.896 INFO:teuthology.orchestra.run.smithi107.stdout:2663 read 0xaf2de00 thru 0xaf35d98 (0x7f99 bytes) 2024-09-11T09:29:22.896 INFO:teuthology.orchestra.run.smithi107.stdout:2667 read 0x4b000 thru 0x56c60 (0xbc61 bytes) 2024-09-11T09:29:22.899 INFO:teuthology.orchestra.run.smithi107.stdout:2668 punch from 0x4e0b600 to 0x4e0e600, (0x3000 bytes) 2024-09-11T09:29:22.902 INFO:teuthology.orchestra.run.smithi107.stdout:2669 trunc from 0xe286600 to 0xb420600 2024-09-11T09:29:22.908 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:22.908+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:22.942 INFO:teuthology.orchestra.run.smithi107.stdout:2670 punch from 0xa34ba00 to 0xa358800, (0xce00 bytes) 2024-09-11T09:29:22.942 INFO:teuthology.orchestra.run.smithi107.stdout:2671 read 0x73e9c00 thru 0x73ed163 (0x3564 bytes) 2024-09-11T09:29:22.942 INFO:teuthology.orchestra.run.smithi107.stdout:2674 read 0x81b1800 thru 0x81b3bfa (0x23fb bytes) 2024-09-11T09:29:22.942 INFO:teuthology.orchestra.run.smithi107.stdout:2678 read 0xb087800 thru 0xb0912b4 (0x9ab5 bytes) 2024-09-11T09:29:22.942 INFO:teuthology.orchestra.run.smithi107.stdout:2679 write 0x1213200 thru 0x12135c6 (0x3c7 bytes) 2024-09-11T09:29:22.943 INFO:teuthology.orchestra.run.smithi107.stdout:2680 read 0x3daac00 thru 0x3db39f1 (0x8df2 bytes) 2024-09-11T09:29:22.943 INFO:teuthology.orchestra.run.smithi107.stdout:2682 punch from 0x3f89800 to 0x3f8f600, (0x5e00 bytes) 2024-09-11T09:29:22.943 INFO:teuthology.orchestra.run.smithi107.stdout:2683 punch from 0x6fac400 to 0x6fae400, (0x2000 bytes) 2024-09-11T09:29:22.943 INFO:teuthology.orchestra.run.smithi107.stdout:2685 read 0x8e86a00 thru 0x8e8c7df (0x5de0 bytes) 2024-09-11T09:29:22.943 INFO:teuthology.orchestra.run.smithi107.stdout:2688 read 0x30f1a00 thru 0x30fbe1a (0xa41b bytes) 2024-09-11T09:29:22.953 INFO:teuthology.orchestra.run.smithi107.stdout:2689 trunc from 0xb420600 to 0x21a6a00 2024-09-11T09:29:22.984 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:22.983+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:2696 punch from 0x8aca00 to 0x8b1000, (0x4600 bytes) 2024-09-11T09:29:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:2698 read 0x13f4a00 thru 0x13ffb97 (0xb198 bytes) 2024-09-11T09:29:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:2700 read 0x1d04600 thru 0x1d0f541 (0xaf42 bytes) 2024-09-11T09:29:23.017 INFO:teuthology.orchestra.run.smithi107.stdout:2702 trunc from 0x21a6a00 to 0xce0a800 2024-09-11T09:29:23.027 INFO:teuthology.orchestra.run.smithi107.stdout:2705 write 0x1301c00 thru 0x130a595 (0x8996 bytes) 2024-09-11T09:29:23.027 INFO:teuthology.orchestra.run.smithi107.stdout:2706 write 0x6843000 thru 0x6850743 (0xd744 bytes) 2024-09-11T09:29:23.028 INFO:teuthology.orchestra.run.smithi107.stdout:2707 punch from 0x8a4b200 to 0x8a57800, (0xc600 bytes) 2024-09-11T09:29:23.032 INFO:teuthology.orchestra.run.smithi107.stdout:2708 write 0x8e92200 thru 0x8e9f74c (0xd54d bytes) 2024-09-11T09:29:23.032 INFO:teuthology.orchestra.run.smithi107.stdout:2709 read 0x2143c00 thru 0x214c07b (0x847c bytes) 2024-09-11T09:29:23.035 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.035 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.043 INFO:teuthology.orchestra.run.smithi107.stdout:2710 trunc from 0xce0a800 to 0x31c1000 2024-09-11T09:29:23.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:23.083 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:23.083+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:23.094 INFO:teuthology.orchestra.run.smithi107.stdout:2712 write 0x3c26400 thru 0x3c2951d (0x311e bytes) 2024-09-11T09:29:23.100 INFO:teuthology.orchestra.run.smithi107.stdout:2713 read 0x81f000 thru 0x8284a2 (0x94a3 bytes) 2024-09-11T09:29:23.101 INFO:teuthology.orchestra.run.smithi107.stdout:2714 read 0x2f81000 thru 0x2f839cd (0x29ce bytes) 2024-09-11T09:29:23.101 INFO:teuthology.orchestra.run.smithi107.stdout:2715 write 0x3227a00 thru 0x32330cb (0xb6cc bytes) 2024-09-11T09:29:23.105 INFO:teuthology.orchestra.run.smithi107.stdout:2716 trunc from 0x3c29600 to 0x6e66000 2024-09-11T09:29:23.128 INFO:teuthology.orchestra.run.smithi107.stdout:2719 trunc from 0x6e66000 to 0x8360c00 2024-09-11T09:29:23.144 INFO:teuthology.orchestra.run.smithi107.stdout:2720 read 0xf74a00 thru 0xf77535 (0x2b36 bytes) 2024-09-11T09:29:23.164 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.164 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:2721 write 0xc396a00 thru 0xc3998db (0x2edc bytes) 2024-09-11T09:29:23.173 INFO:teuthology.orchestra.run.smithi107.stdout:2725 write 0x3f19a00 thru 0x3f26338 (0xc939 bytes) 2024-09-11T09:29:23.173 INFO:teuthology.orchestra.run.smithi107.stdout:2726 punch from 0x9464e00 to 0x9468800, (0x3a00 bytes) 2024-09-11T09:29:23.174 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2727 read 0x5055400 thru 0x505b359 (0x5f5a bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2728 read 0x7992400 thru 0x799f234 (0xce35 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2730 punch from 0x8cdde00 to 0x8ce8a00, (0xac00 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2731 write 0x7eeaa00 thru 0x7ef0db4 (0x63b5 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2736 write 0x5b91600 thru 0x5b9ab08 (0x9509 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2737 punch from 0x1300600 to 0x1304400, (0x3e00 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2741 write 0x63a4c00 thru 0x63b2c06 (0xe007 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2743 read 0x4912400 thru 0x4916afd (0x46fe bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2745 read 0x70d1000 thru 0x70d3b87 (0x2b88 bytes) 2024-09-11T09:29:23.176 INFO:teuthology.orchestra.run.smithi107.stdout:2747 read 0xadc0600 thru 0xadc6296 (0x5c97 bytes) 2024-09-11T09:29:23.177 INFO:teuthology.orchestra.run.smithi107.stdout:2749 clone 16 order 22 su 32768 sc 10 2024-09-11T09:29:23.221 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:23.222+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:23.285 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:23.285+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:23.293 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.294 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.302 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:23.386 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:23.387+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:23.404 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.404 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:23.487 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:23.488+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:23.514 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.514 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.523 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:23.588 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:23.588+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:23.626 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.626 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.634 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:23.689 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:23.689+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:23.737 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.737 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:23.790 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:23.791+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:23.820 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T09:29:23.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T09:29:23.893 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:23.893 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:23.902 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:23.918 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:23.919+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:23.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:23.992 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:23.992+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:24.008 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:24.008 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:24.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:24.101 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:24.102+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:24.103 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:24.103 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:24.121 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:24.139 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:24.139 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:24.157 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:24.157 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:24.169 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:24.169 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:24.178 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:24.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:24.195 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:24.222 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:24.222+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:24.269 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-11T09:29:24.294 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:24.295+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:24.325 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:24.325 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:24.326 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:24.326 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:24.329 INFO:teuthology.orchestra.run.smithi107.stdout:2750 punch from 0xbdb5800 to 0xbdc4a00, (0xf200 bytes) 2024-09-11T09:29:24.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:24.345 INFO:teuthology.orchestra.run.smithi107.stdout:2752 punch from 0x2b13a00 to 0x2b14a00, (0x1000 bytes) 2024-09-11T09:29:24.350 INFO:teuthology.orchestra.run.smithi107.stdout:2753 read 0x31bc00 thru 0x329c97 (0xe098 bytes) 2024-09-11T09:29:24.351 INFO:teuthology.orchestra.run.smithi107.stdout:2756 punch from 0x20cd000 to 0x20d8400, (0xb400 bytes) 2024-09-11T09:29:24.382 INFO:teuthology.orchestra.run.smithi107.stdout:2757 punch from 0x4739c00 to 0x473b000, (0x1400 bytes) 2024-09-11T09:29:24.388 INFO:teuthology.orchestra.run.smithi107.stdout:2759 read 0x1ee8400 thru 0x1eeda81 (0x5682 bytes) 2024-09-11T09:29:24.388 INFO:teuthology.orchestra.run.smithi107.stdout:2762 punch from 0xcc8a00 to 0xcd0e00, (0x8400 bytes) 2024-09-11T09:29:24.395 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:24.396+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:24.409 INFO:teuthology.orchestra.run.smithi107.stdout:2763 write 0xe31a800 thru 0xe3235a0 (0x8da1 bytes) 2024-09-11T09:29:24.424 INFO:teuthology.orchestra.run.smithi107.stdout:2765 trunc from 0xe323600 to 0x1e6a200 2024-09-11T09:29:24.475 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4835 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.491224334, 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033722624999999999, 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4835 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7800000000000002e-07 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983045+0000", 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.657e-06 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983049+0000", 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3420000000000001e-06 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983052+0000", 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.74e-06 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016669+0000", 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033617247000000003 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016711+0000", 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1149999999999997e-05 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016766+0000", 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5111000000000001e-05 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.491168484999999, 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.491146710000001, 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.491100665999999, 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.491088996, 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.490215587, 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489852111999999, 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:24.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489800727, 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.509 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:24.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:24.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489765026000001, 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:24.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:24.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.517 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489742989, 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:24.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:24.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489707404000001, 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:24.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.526 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489647060999999, 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:24.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:24.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:24.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489598377, 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.531 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:24.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489541913, 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489522358, 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:24.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489474595000001, 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:24.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489360856999999, 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:24.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:24.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489342112999999, 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:24.553 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:24.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489303123999999, 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:24.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.489280017, 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.563 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:24.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:24.567 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:24.570 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:24.569+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:24.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2766 write 0xbbad200 thru 0xbbb5111 (0x7f12 bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2767 write 0x42e3800 thru 0x42eacbf (0x74c0 bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2768 write 0xdfe7400 thru 0xdfeac6a (0x386b bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2770 punch from 0xe6d200 to 0xe71000, (0x3e00 bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2771 write 0x6d14a00 thru 0x6d1cce9 (0x82ea bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2772 write 0x314b000 thru 0x31506bc (0x56bd bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2776 punch from 0x6486200 to 0x6489600, (0x3400 bytes) 2024-09-11T09:29:24.571 INFO:teuthology.orchestra.run.smithi107.stdout:2777 write 0xa576800 thru 0xa57b193 (0x4994 bytes) 2024-09-11T09:29:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:2779 punch from 0xace9c00 to 0xacf0a00, (0x6e00 bytes) 2024-09-11T09:29:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:2785 write 0x61dda00 thru 0x61ea825 (0xce26 bytes) 2024-09-11T09:29:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:2788 write 0x97da400 thru 0x97dc8a0 (0x24a1 bytes) 2024-09-11T09:29:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:2790 write 0xd6b7c00 thru 0xd6be5b4 (0x69b5 bytes) 2024-09-11T09:29:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:29:24.572 INFO:teuthology.orchestra.run.smithi107.stdout:2793 write 0xa6e0000 thru 0xa6e17c3 (0x17c4 bytes) 2024-09-11T09:29:24.576 INFO:teuthology.orchestra.run.smithi107.stdout:2795 trunc from 0xdfeae00 to 0x6dc1c00 2024-09-11T09:29:24.604 INFO:teuthology.orchestra.run.smithi107.stdout:2796 read 0x27db800 thru 0x27e92a2 (0xdaa3 bytes) 2024-09-11T09:29:24.614 INFO:teuthology.orchestra.run.smithi107.stdout:2797 trunc from 0x6dc1c00 to 0xe6fce00 2024-09-11T09:29:24.624 INFO:teuthology.orchestra.run.smithi107.stdout:2798 write 0x4d0d400 thru 0x4d0e411 (0x1012 bytes) 2024-09-11T09:29:24.637 INFO:teuthology.orchestra.run.smithi107.stdout:2800 trunc from 0xe6fce00 to 0x3b1a400 2024-09-11T09:29:24.669 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:24.670+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:24.673 INFO:teuthology.orchestra.run.smithi107.stdout:2801 write 0x8502400 thru 0x8503ee7 (0x1ae8 bytes) 2024-09-11T09:29:24.674 INFO:teuthology.orchestra.run.smithi107.stdout:2802 read 0x4502200 thru 0x450d564 (0xb365 bytes) 2024-09-11T09:29:24.674 INFO:teuthology.orchestra.run.smithi107.stdout:2803 read 0x39fc800 thru 0x3a0b879 (0xf07a bytes) 2024-09-11T09:29:24.675 INFO:teuthology.orchestra.run.smithi107.stdout:2804 read 0x4e12e00 thru 0x4e19209 (0x640a bytes) 2024-09-11T09:29:24.675 INFO:teuthology.orchestra.run.smithi107.stdout:2805 punch from 0x3e6d200 to 0x3e76600, (0x9400 bytes) 2024-09-11T09:29:24.687 INFO:teuthology.orchestra.run.smithi107.stdout:2808 write 0xb301000 thru 0xb3072df (0x62e0 bytes) 2024-09-11T09:29:24.688 INFO:teuthology.orchestra.run.smithi107.stdout:2810 read 0x9cf7400 thru 0x9cfd3f3 (0x5ff4 bytes) 2024-09-11T09:29:24.688 INFO:teuthology.orchestra.run.smithi107.stdout:2811 write 0xa2db800 thru 0xa2ddd5c (0x255d bytes) 2024-09-11T09:29:24.695 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:24.695 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:24.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.695 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:11678 3.12 3:4a7751b6:::rbd_header.1106e0c0ccd2:head [call lock.lock in=76b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-11T09:29:24.695 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:24.696046+0000", 2024-09-11T09:29:24.695 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.00027433699999999999, 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00030019399999999999, 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 11678 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696046+0000", 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696046+0000", 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696046+0000", 2024-09-11T09:29:24.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9200000000000001e-07 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696050+0000", 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7390000000000001e-06 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696059+0000", 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9290000000000004e-06 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696064+0000", 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6489999999999999e-06 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696097+0000", 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3495999999999999e-05 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:24.696141+0000", 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3396999999999998e-05 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:24.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:24.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:24.700 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:29:24.700 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 1 2024-09-11T09:29:24.700 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:24.702 INFO:teuthology.orchestra.run.smithi107.stdout:2812 write 0xcf11400 thru 0xcf1c5c9 (0xb1ca bytes) 2024-09-11T09:29:24.703 INFO:teuthology.orchestra.run.smithi107.stdout:2813 write 0x694ca00 thru 0x695a5a3 (0xdba4 bytes) 2024-09-11T09:29:24.703 INFO:teuthology.orchestra.run.smithi107.stdout:2816 write 0xa3de00 thru 0xa49af0 (0xbcf1 bytes) 2024-09-11T09:29:24.703 INFO:teuthology.orchestra.run.smithi107.stdout:2817 write 0xba89600 thru 0xba90bf2 (0x75f3 bytes) 2024-09-11T09:29:24.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:24.717 INFO:teuthology.orchestra.run.smithi107.stdout:2818 trunc from 0xcf1c600 to 0x10aa000 2024-09-11T09:29:24.749 INFO:teuthology.orchestra.run.smithi107.stdout:2819 read 0x738a00 thru 0x747275 (0xe876 bytes) 2024-09-11T09:29:24.767 INFO:teuthology.orchestra.run.smithi107.stdout:2820 write 0x76f1e00 thru 0x76f800e (0x620f bytes) 2024-09-11T09:29:24.770 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:24.771+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:24.772 INFO:teuthology.orchestra.run.smithi107.stdout:2824 read 0x6e5ac00 thru 0x6e646cd (0x9ace bytes) 2024-09-11T09:29:24.782 INFO:teuthology.orchestra.run.smithi107.stdout:2826 write 0x99dfe00 thru 0x99e29bd (0x2bbe bytes) 2024-09-11T09:29:24.798 INFO:teuthology.orchestra.run.smithi107.stdout:2830 write 0xa51c800 thru 0xa529cfc (0xd4fd bytes) 2024-09-11T09:29:24.804 INFO:teuthology.orchestra.run.smithi107.stdout:2834 punch from 0x22a8200 to 0x22aae00, (0x2c00 bytes) 2024-09-11T09:29:24.806 INFO:teuthology.orchestra.run.smithi107.stdout:2835 read 0x2d37600 thru 0x2d38922 (0x1323 bytes) 2024-09-11T09:29:24.823 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T09:29:24.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:29:24.831 INFO:teuthology.orchestra.run.smithi107.stdout:2836 write 0xec74c00 thru 0xec82675 (0xda76 bytes) 2024-09-11T09:29:24.841 INFO:teuthology.orchestra.run.smithi107.stdout:2837 trunc from 0xec82800 to 0x61a9c00 2024-09-11T09:29:24.842 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T09:29:24.843 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:29:24.871 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:24.871+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:24.872 INFO:teuthology.orchestra.run.smithi107.stdout:2839 trunc from 0x61a9c00 to 0x94a4a00 2024-09-11T09:29:24.873 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:24.873 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:24.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:24.891 INFO:teuthology.orchestra.run.smithi107.stdout:2840 trunc from 0x94a4a00 to 0x215fe00 2024-09-11T09:29:24.928 INFO:teuthology.orchestra.run.smithi107.stdout:2842 write 0x620d600 thru 0x621acde (0xd6df bytes) 2024-09-11T09:29:24.929 INFO:teuthology.orchestra.run.smithi107.stdout:2843 read 0x1f50400 thru 0x1f527f3 (0x23f4 bytes) 2024-09-11T09:29:24.929 INFO:teuthology.orchestra.run.smithi107.stdout:2844 read 0x3a3fe00 thru 0x3a41913 (0x1b14 bytes) 2024-09-11T09:29:24.929 INFO:teuthology.orchestra.run.smithi107.stdout:2845 punch from 0x1563400 to 0x1564000, (0xc00 bytes) 2024-09-11T09:29:24.932 INFO:teuthology.orchestra.run.smithi107.stdout:2846 trunc from 0x621ae00 to 0x79e8a00 2024-09-11T09:29:24.949 INFO:teuthology.orchestra.run.smithi107.stdout:2848 read 0x6758800 thru 0x675cefc (0x46fd bytes) 2024-09-11T09:29:24.958 INFO:teuthology.orchestra.run.smithi107.stdout:2850 trunc from 0x79e8a00 to 0x1a4f000 2024-09-11T09:29:24.972 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:24.972+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:25.001 INFO:teuthology.orchestra.run.smithi107.stdout:2851 write 0xc04c600 thru 0xc05375b (0x715c bytes) 2024-09-11T09:29:25.001 INFO:teuthology.orchestra.run.smithi107.stdout:2852 write 0xb22a00 thru 0xb29716 (0x6d17 bytes) 2024-09-11T09:29:25.002 INFO:teuthology.orchestra.run.smithi107.stdout:2853 read 0x84e4000 thru 0x84e5e9b (0x1e9c bytes) 2024-09-11T09:29:25.002 INFO:teuthology.orchestra.run.smithi107.stdout:2854 write 0x4119a00 thru 0x41201a0 (0x67a1 bytes) 2024-09-11T09:29:25.002 INFO:teuthology.orchestra.run.smithi107.stdout:2857 read 0x3b97000 thru 0x3ba49c6 (0xd9c7 bytes) 2024-09-11T09:29:25.002 INFO:teuthology.orchestra.run.smithi107.stdout:2858 read 0x29bc200 thru 0x29cb57c (0xf37d bytes) 2024-09-11T09:29:25.002 INFO:teuthology.orchestra.run.smithi107.stdout:2859 write 0x1ecf800 thru 0x1eddafd (0xe2fe bytes) 2024-09-11T09:29:25.003 INFO:teuthology.orchestra.run.smithi107.stdout:2860 read 0x502ea00 thru 0x503b2b2 (0xc8b3 bytes) 2024-09-11T09:29:25.003 INFO:teuthology.orchestra.run.smithi107.stdout:2863 write 0x53d600 thru 0x5487e8 (0xb1e9 bytes) 2024-09-11T09:29:25.003 INFO:teuthology.orchestra.run.smithi107.stdout:2866 read 0x1504000 thru 0x1508078 (0x4079 bytes) 2024-09-11T09:29:25.004 INFO:teuthology.orchestra.run.smithi107.stdout:2867 write 0x9b86800 thru 0x9b94aeb (0xe2ec bytes) 2024-09-11T09:29:25.004 INFO:teuthology.orchestra.run.smithi107.stdout:2868 write 0xaf77c00 thru 0xaf8143e (0x983f bytes) 2024-09-11T09:29:25.012 INFO:teuthology.orchestra.run.smithi107.stdout:2869 trunc from 0xc053800 to 0x4dd5200 2024-09-11T09:29:25.054 INFO:teuthology.orchestra.run.smithi107.stdout:2872 write 0x92c0600 thru 0x92ca22c (0x9c2d bytes) 2024-09-11T09:29:25.054 INFO:teuthology.orchestra.run.smithi107.stdout:2873 write 0x577c400 thru 0x5789b19 (0xd71a bytes) 2024-09-11T09:29:25.058 INFO:teuthology.orchestra.run.smithi107.stdout:2874 trunc from 0x92ca400 to 0x5e80200 2024-09-11T09:29:25.072 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:25.073+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:25.087 INFO:teuthology.orchestra.run.smithi107.stdout:2875 trunc from 0x5e80200 to 0xc784200 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.418449889000001, 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:25.091 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:25.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:25.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.097 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1715 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213226225, 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022344029000000001, 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.098 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1715 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.96e-07 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876407+0000", 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.871e-06 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876414+0000", 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4679999999999997e-06 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876419+0000", 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1939999999999997e-06 2024-09-11T09:29:25.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898563+0000", 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022144646 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898643+0000", 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9900000000000004e-05 2024-09-11T09:29:25.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898749+0000", 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105554 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1716 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:25.103 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213205964, 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022552065999999999, 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1716 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876426+0000", 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.77e-07 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876432+0000", 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0079999999999999e-06 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876438+0000", 2024-09-11T09:29:25.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4999999999999996e-06 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876443+0000", 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.259e-06 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898781+0000", 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022337308 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898875+0000", 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4388000000000005e-05 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898977+0000", 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010222599999999999 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1717 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:25.108 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213181059, 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022753961, 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1717 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:25.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876451+0000", 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876452+0000", 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.905e-06 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876459+0000", 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4980000000000003e-06 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876464+0000", 2024-09-11T09:29:25.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1909999999999999e-06 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899008+0000", 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022544281999999999 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899093+0000", 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4505000000000005e-05 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899204+0000", 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111049 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1718 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213160394999999, 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02296196, 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1718 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876473+0000", 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9259999999999999e-06 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876480+0000", 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876485+0000", 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2159999999999997e-06 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899235+0000", 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022750631 2024-09-11T09:29:25.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899321+0000", 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5958999999999995e-05 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899433+0000", 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011126899999999999 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1719 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 262144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213119978000002, 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023149355, 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1719 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876512+0000", 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.44e-07 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876514+0000", 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876522+0000", 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5000000000000002e-06 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876527+0000", 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.587e-06 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899467+0000", 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022939975000000001 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899552+0000", 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5131999999999997e-05 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899660+0000", 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108137 2024-09-11T09:29:25.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1720 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213096388, 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023387654000000001, 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.123 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1720 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.63e-07 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876537+0000", 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1040000000000001e-06 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876544+0000", 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7800000000000003e-06 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876549+0000", 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2549999999999997e-06 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899706+0000", 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023156708000000002 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899801+0000", 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4826000000000003e-05 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899922+0000", 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121518 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.127 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1760 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 12288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213075495999998, 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023618561999999999, 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1760 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876558+0000", 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1670000000000002e-06 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876566+0000", 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3810000000000001e-06 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876571+0000", 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3029999999999999e-06 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899963+0000", 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023392445000000001 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900061+0000", 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7495000000000001e-05 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900174+0000", 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113391 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1761 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213053695999999, 2024-09-11T09:29:25.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023824222999999999, 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1761 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876578+0000", 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:25.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876580+0000", 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9539999999999998e-06 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876586+0000", 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876591+0000", 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900220+0000", 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023628546 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900317+0000", 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7168000000000002e-05 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900402+0000", 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4701000000000001e-05 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1762 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 57344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213033441, 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024033523000000001, 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1762 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876600+0000", 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.052e-06 2024-09-11T09:29:25.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876607+0000", 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876612+0000", 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9259999999999999e-06 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900448+0000", 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023836570000000001 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900534+0000", 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5314000000000003e-05 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900631+0000", 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7676000000000005e-05 2024-09-11T09:29:25.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1763 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 79872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.213013015000001, 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024166084000000001, 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.143 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1763 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876619+0000", 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6400000000000003e-07 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876621+0000", 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0949999999999998e-06 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876629+0000", 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8380000000000008e-06 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876635+0000", 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3199999999999999e-06 2024-09-11T09:29:25.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900675+0000", 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024040611 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900724+0000", 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8097999999999999e-05 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900784+0000", 2024-09-11T09:29:25.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0658000000000001e-05 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1764 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212989778000001, 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024356692999999999, 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1764 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876642+0000", 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876644+0000", 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.041e-06 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876651+0000", 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6309999999999997e-06 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876656+0000", 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999999e-06 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900814+0000", 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024158129 2024-09-11T09:29:25.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900906+0000", 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2618e-05 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900998+0000", 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1624000000000002e-05 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1765 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212969048000001, 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024574166000000001, 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1765 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.153 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:25.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.39e-07 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876664+0000", 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9520000000000001e-06 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876671+0000", 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3670000000000002e-06 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876676+0000", 2024-09-11T09:29:25.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901029+0000", 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024353631000000001 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901109+0000", 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9951999999999996e-05 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901236+0000", 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012697000000000001 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212949258999998, 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:25.157 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:25.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:25.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212929027000001, 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.162 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:25.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212902901, 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:25.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212880616, 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:25.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212857176, 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:25.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.180 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212837328999999, 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:25.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.212804951999999, 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.185 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:25.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:25.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.189 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:25.191 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:25.191+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:25.192 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2877 trunc from 0xc784200 to 0xdfdfc00 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2879 read 0x7fb9c00 thru 0x7fc96f2 (0xfaf3 bytes) 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2880 punch from 0xd5d4000 to 0xd5df600, (0xb600 bytes) 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2883 write 0xdb6bc00 thru 0xdb71111 (0x5512 bytes) 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2884 write 0x2729a00 thru 0x273115b (0x775c bytes) 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2887 write 0x906e200 thru 0x9075706 (0x7507 bytes) 2024-09-11T09:29:25.193 INFO:teuthology.orchestra.run.smithi107.stdout:2888 read 0x2e6ee00 thru 0x2e77898 (0x8a99 bytes) 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2889 read 0xa1bc000 thru 0xa1cb068 (0xf069 bytes) 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2890 read 0x8ef2800 thru 0x8efff03 (0xd704 bytes) 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2892 write 0xc179a00 thru 0xc1810a6 (0x76a7 bytes) 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2893 trunc from 0xdfdfc00 to 0x19b3e00 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2895 punch from 0x556a00 to 0x558400, (0x1a00 bytes) 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2903 write 0x3f8b000 thru 0x3f94df6 (0x9df7 bytes) 2024-09-11T09:29:25.194 INFO:teuthology.orchestra.run.smithi107.stdout:2905 trunc from 0x3f94e00 to 0x8edfa00 2024-09-11T09:29:25.212 INFO:teuthology.orchestra.run.smithi107.stdout:2907 write 0xaf7dc00 thru 0xaf87fac (0xa3ad bytes) 2024-09-11T09:29:25.212 INFO:teuthology.orchestra.run.smithi107.stdout:2908 punch from 0x53d800 to 0x541200, (0x3a00 bytes) 2024-09-11T09:29:25.217 INFO:teuthology.orchestra.run.smithi107.stdout:2909 read 0x44d5200 thru 0x44dd2b4 (0x80b5 bytes) 2024-09-11T09:29:25.221 INFO:teuthology.orchestra.run.smithi107.stdout:2911 read 0x431bc00 thru 0x4323f98 (0x8399 bytes) 2024-09-11T09:29:25.221 INFO:teuthology.orchestra.run.smithi107.stdout:2912 read 0x30fd200 thru 0x310be7e (0xec7f bytes) 2024-09-11T09:29:25.221 INFO:teuthology.orchestra.run.smithi107.stdout:2914 read 0x672de00 thru 0x6734f21 (0x7122 bytes) 2024-09-11T09:29:25.223 INFO:teuthology.orchestra.run.smithi107.stdout:2917 trunc from 0xaf88000 to 0xc5c4a00 2024-09-11T09:29:25.234 INFO:teuthology.orchestra.run.smithi107.stdout:2919 trunc from 0xc5c4a00 to 0xb535a00 2024-09-11T09:29:25.258 INFO:teuthology.orchestra.run.smithi107.stdout:2921 trunc from 0xb535a00 to 0x456b400 2024-09-11T09:29:25.266 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:29:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:29:25.273 INFO:teuthology.orchestra.run.smithi107.stdout:2924 write 0x1599200 thru 0x159b785 (0x2586 bytes) 2024-09-11T09:29:25.279 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-11T09:29:24.824661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":388459243,"num_objects":811,"num_object_clones":226,"num_object_copies":1622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":811,"num_whiteouts":241,"num_read":121620,"num_read_kb":475420,"num_write":10630,"num_write_kb":110793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":148526,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8976,"ondisk_log_size":8976,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":234980,"kb_used_data":17988,"kb_used_omap":7,"kb_used_meta":216888,"kb_avail":471624220,"statfs":{"total":483183820800,"available":482943201280,"internally_reserved":0,"allocated":18419712,"data_stored":116405474,"data_compressed":4541938,"data_compressed_allocated":8593408,"data_compressed_original":109425664,"omap_allocated":7973,"internal_metadata":222093531},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20144141,"num_objects":172,"num_object_clones":26,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":21,"num_read":23024,"num_read_kb":63394,"num_write":3795,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017373"},"pg_stats":[{"pgid":"3.1f","version":"47'169","reported_seq":1304,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.618965+0000","last_change":"2024-09-11T09:28:48.712900+0000","last_active":"2024-09-11T09:29:21.618965+0000","last_peered":"2024-09-11T09:29:21.618965+0000","last_clean":"2024-09-11T09:29:21.618965+0000","last_became_active":"2024-09-11T09:28:46.699073+0000","last_became_peered":"2024-09-11T09:28:46.699073+0000","last_unstale":"2024-09-11T09:29:21.618965+0000","last_undegraded":"2024-09-11T09:29:21.618965+0000","last_fullsized":"2024-09-11T09:29:21.618965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:03.561937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012367610000000001,"stat_sum":{"num_bytes":17482768,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":1112,"num_read_kb":9691,"num_write":173,"num_write_kb":9430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"49'307","reported_seq":2811,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.153098+0000","last_change":"2024-09-11T09:28:52.719933+0000","last_active":"2024-09-11T09:29:23.153098+0000","last_peered":"2024-09-11T09:29:23.153098+0000","last_clean":"2024-09-11T09:29:23.153098+0000","last_became_active":"2024-09-11T09:28:52.715808+0000","last_became_peered":"2024-09-11T09:28:52.715808+0000","last_unstale":"2024-09-11T09:29:23.153098+0000","last_undegraded":"2024-09-11T09:29:23.153098+0000","last_fullsized":"2024-09-11T09:29:23.153098+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:27:42.284486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001676,"stat_sum":{"num_bytes":12437337,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3658,"num_read_kb":18869,"num_write":303,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"49'390","reported_seq":1441,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.829014+0000","last_change":"2024-09-11T09:28:48.709187+0000","last_active":"2024-09-11T09:29:22.829014+0000","last_peered":"2024-09-11T09:29:22.829014+0000","last_clean":"2024-09-11T09:29:22.829014+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:29:22.829014+0000","last_undegraded":"2024-09-11T09:29:22.829014+0000","last_fullsized":"2024-09-11T09:29:22.829014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:46.255889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015577869999999999,"stat_sum":{"num_bytes":22194763,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2441,"num_read_kb":19626,"num_write":606,"num_write_kb":9462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"49'154","reported_seq":5357,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.975838+0000","last_change":"2024-09-11T09:28:48.708858+0000","last_active":"2024-09-11T09:29:22.975838+0000","last_peered":"2024-09-11T09:29:22.975838+0000","last_clean":"2024-09-11T09:29:22.975838+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:29:22.975838+0000","last_undegraded":"2024-09-11T09:29:22.975838+0000","last_fullsized":"2024-09-11T09:29:22.975838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:17.234093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013196460000000001,"stat_sum":{"num_bytes":11452456,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":5210,"num_read_kb":7606,"num_write":144,"num_write_kb":2075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"47'120","reported_seq":1526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.728268+0000","last_change":"2024-09-11T09:28:48.712945+0000","last_active":"2024-09-11T09:29:21.728268+0000","last_peered":"2024-09-11T09:29:21.728268+0000","last_clean":"2024-09-11T09:29:21.728268+0000","last_became_active":"2024-09-11T09:28:46.702512+0000","last_became_peered":"2024-09-11T09:28:46.702512+0000","last_unstale":"2024-09-11T09:29:21.728268+0000","last_undegraded":"2024-09-11T09:29:21.728268+0000","last_fullsized":"2024-09-11T09:29:21.728268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:37:42.492275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012800909999999999,"stat_sum":{"num_bytes":3618304,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":1358,"num_read_kb":8030,"num_write":115,"num_write_kb":1455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"48'232","reported_seq":3060,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.166706+0000","last_change":"2024-09-11T09:28:48.704277+0000","last_active":"2024-09-11T09:29:23.166706+0000","last_peered":"2024-09-11T09:29:23.166706+0000","last_clean":"2024-09-11T09:29:23.166706+0000","last_became_active":"2024-09-11T09:28:46.704959+0000","last_became_peered":"2024-09-11T09:28:46.704959+0000","last_unstale":"2024-09-11T09:29:23.166706+0000","last_undegraded":"2024-09-11T09:29:23.166706+0000","last_fullsized":"2024-09-11T09:29:23.166706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:13:26.702896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081867,"stat_sum":{"num_bytes":3345531,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":3268,"num_read_kb":11603,"num_write":287,"num_write_kb":962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"49'194","reported_seq":551,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.972664+0000","last_change":"2024-09-11T09:28:48.699507+0000","last_active":"2024-09-11T09:29:22.972664+0000","last_peered":"2024-09-11T09:29:22.972664+0000","last_clean":"2024-09-11T09:29:22.972664+0000","last_became_active":"2024-09-11T09:28:46.701636+0000","last_became_peered":"2024-09-11T09:28:46.701636+0000","last_unstale":"2024-09-11T09:29:22.972664+0000","last_undegraded":"2024-09-11T09:29:22.972664+0000","last_fullsized":"2024-09-11T09:29:22.972664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:45:18.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058014000000000004,"stat_sum":{"num_bytes":3833474,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":466,"num_read_kb":2749,"num_write":211,"num_write_kb":3202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"47'215","reported_seq":904,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.655623+0000","last_change":"2024-09-11T09:28:52.717462+0000","last_active":"2024-09-11T09:29:21.655623+0000","last_peered":"2024-09-11T09:29:21.655623+0000","last_clean":"2024-09-11T09:29:21.655623+0000","last_became_active":"2024-09-11T09:28:52.717165+0000","last_became_peered":"2024-09-11T09:28:52.717165+0000","last_unstale":"2024-09-11T09:29:21.655623+0000","last_undegraded":"2024-09-11T09:29:21.655623+0000","last_fullsized":"2024-09-11T09:29:21.655623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:59.930098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226532,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":1206,"num_read_kb":7262,"num_write":288,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"47'203","reported_seq":3411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.542390+0000","last_change":"2024-09-11T09:28:48.713192+0000","last_active":"2024-09-11T09:29:21.542390+0000","last_peered":"2024-09-11T09:29:21.542390+0000","last_clean":"2024-09-11T09:29:21.542390+0000","last_became_active":"2024-09-11T09:28:46.702717+0000","last_became_peered":"2024-09-11T09:28:46.702717+0000","last_unstale":"2024-09-11T09:29:21.542390+0000","last_undegraded":"2024-09-11T09:29:21.542390+0000","last_fullsized":"2024-09-11T09:29:21.542390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:45:41.787524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001366531,"stat_sum":{"num_bytes":8125637,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":3594,"num_read_kb":16530,"num_write":263,"num_write_kb":8540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"49'545","reported_seq":5763,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.207538+0000","last_change":"2024-09-11T09:28:48.709601+0000","last_active":"2024-09-11T09:29:23.207538+0000","last_peered":"2024-09-11T09:29:23.207538+0000","last_clean":"2024-09-11T09:29:23.207538+0000","last_became_active":"2024-09-11T09:28:46.699041+0000","last_became_peered":"2024-09-11T09:28:46.699041+0000","last_unstale":"2024-09-11T09:29:23.207538+0000","last_undegraded":"2024-09-11T09:29:23.207538+0000","last_fullsized":"2024-09-11T09:29:23.207538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:27.083097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017410150000000001,"stat_sum":{"num_bytes":7954049,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":9083,"num_read_kb":23429,"num_write":642,"num_write_kb":2017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"49'151","reported_seq":1515,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.118722+0000","last_change":"2024-09-11T09:28:48.699698+0000","last_active":"2024-09-11T09:29:23.118722+0000","last_peered":"2024-09-11T09:29:23.118722+0000","last_clean":"2024-09-11T09:29:23.118722+0000","last_became_active":"2024-09-11T09:28:46.700365+0000","last_became_peered":"2024-09-11T09:28:46.700365+0000","last_unstale":"2024-09-11T09:29:23.118722+0000","last_undegraded":"2024-09-11T09:29:23.118722+0000","last_fullsized":"2024-09-11T09:29:23.118722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:03.735174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014058110000000001,"stat_sum":{"num_bytes":9364049,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":1346,"num_read_kb":3414,"num_write":142,"num_write_kb":866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"49'124","reported_seq":375,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.064909+0000","last_change":"2024-09-11T09:28:48.698947+0000","last_active":"2024-09-11T09:29:23.064909+0000","last_peered":"2024-09-11T09:29:23.064909+0000","last_clean":"2024-09-11T09:29:23.064909+0000","last_became_active":"2024-09-11T09:28:46.703111+0000","last_became_peered":"2024-09-11T09:28:46.703111+0000","last_unstale":"2024-09-11T09:29:23.064909+0000","last_undegraded":"2024-09-11T09:29:23.064909+0000","last_fullsized":"2024-09-11T09:29:23.064909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:20.769364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063234499999999998,"stat_sum":{"num_bytes":2714639,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":189,"num_read_kb":2133,"num_write":121,"num_write_kb":727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816167+0000","last_change":"2024-09-11T09:28:38.615508+0000","last_active":"2024-09-11T09:29:22.816167+0000","last_peered":"2024-09-11T09:29:22.816167+0000","last_clean":"2024-09-11T09:29:22.816167+0000","last_became_active":"2024-09-11T09:28:36.535007+0000","last_became_peered":"2024-09-11T09:28:36.535007+0000","last_unstale":"2024-09-11T09:29:22.816167+0000","last_undegraded":"2024-09-11T09:29:22.816167+0000","last_fullsized":"2024-09-11T09:29:22.816167+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:46.010169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068309000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'124","reported_seq":1442,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.716770+0000","last_change":"2024-09-11T09:28:48.712155+0000","last_active":"2024-09-11T09:29:21.716770+0000","last_peered":"2024-09-11T09:29:21.716770+0000","last_clean":"2024-09-11T09:29:21.716770+0000","last_became_active":"2024-09-11T09:28:46.700016+0000","last_became_peered":"2024-09-11T09:28:46.700016+0000","last_unstale":"2024-09-11T09:29:21.716770+0000","last_undegraded":"2024-09-11T09:29:21.716770+0000","last_fullsized":"2024-09-11T09:29:21.716770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:38:46.091365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051448599999999998,"stat_sum":{"num_bytes":14400000,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":1267,"num_read_kb":5262,"num_write":117,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816192+0000","last_change":"2024-09-11T09:28:38.462223+0000","last_active":"2024-09-11T09:29:22.816192+0000","last_peered":"2024-09-11T09:29:22.816192+0000","last_clean":"2024-09-11T09:29:22.816192+0000","last_became_active":"2024-09-11T09:28:36.455841+0000","last_became_peered":"2024-09-11T09:28:36.455841+0000","last_unstale":"2024-09-11T09:29:22.816192+0000","last_undegraded":"2024-09-11T09:29:22.816192+0000","last_fullsized":"2024-09-11T09:29:22.816192+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:26.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'121","reported_seq":3518,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.037039+0000","last_change":"2024-09-11T09:28:52.719363+0000","last_active":"2024-09-11T09:29:23.037039+0000","last_peered":"2024-09-11T09:29:23.037039+0000","last_clean":"2024-09-11T09:29:23.037039+0000","last_became_active":"2024-09-11T09:28:52.719015+0000","last_became_peered":"2024-09-11T09:28:52.719015+0000","last_unstale":"2024-09-11T09:29:23.037039+0000","last_undegraded":"2024-09-11T09:29:23.037039+0000","last_fullsized":"2024-09-11T09:29:23.037039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:46.018964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7513600,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":3355,"num_read_kb":9852,"num_write":116,"num_write_kb":2383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816107+0000","last_change":"2024-09-11T09:28:38.615243+0000","last_active":"2024-09-11T09:29:22.816107+0000","last_peered":"2024-09-11T09:29:22.816107+0000","last_clean":"2024-09-11T09:29:22.816107+0000","last_became_active":"2024-09-11T09:28:36.453319+0000","last_became_peered":"2024-09-11T09:28:36.453319+0000","last_unstale":"2024-09-11T09:29:22.816107+0000","last_undegraded":"2024-09-11T09:29:22.816107+0000","last_fullsized":"2024-09-11T09:29:22.816107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:41.513882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047127699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'161","reported_seq":842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.007384+0000","last_change":"2024-09-11T09:28:52.718252+0000","last_active":"2024-09-11T09:29:23.007384+0000","last_peered":"2024-09-11T09:29:23.007384+0000","last_clean":"2024-09-11T09:29:23.007384+0000","last_became_active":"2024-09-11T09:28:52.717825+0000","last_became_peered":"2024-09-11T09:28:52.717825+0000","last_unstale":"2024-09-11T09:29:23.007384+0000","last_undegraded":"2024-09-11T09:29:23.007384+0000","last_fullsized":"2024-09-11T09:29:23.007384+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:18:32.603140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729355,"stat_sum":{"num_bytes":10445824,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":638,"num_read_kb":9778,"num_write":155,"num_write_kb":3807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816127+0000","last_change":"2024-09-11T09:28:38.462264+0000","last_active":"2024-09-11T09:29:22.816127+0000","last_peered":"2024-09-11T09:29:22.816127+0000","last_clean":"2024-09-11T09:29:22.816127+0000","last_became_active":"2024-09-11T09:28:36.454444+0000","last_became_peered":"2024-09-11T09:28:36.454444+0000","last_unstale":"2024-09-11T09:29:22.816127+0000","last_undegraded":"2024-09-11T09:29:22.816127+0000","last_fullsized":"2024-09-11T09:29:22.816127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:56.449791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050275600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"49'196","reported_seq":2724,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.157303+0000","last_change":"2024-09-11T09:28:52.729839+0000","last_active":"2024-09-11T09:29:23.157303+0000","last_peered":"2024-09-11T09:29:23.157303+0000","last_clean":"2024-09-11T09:29:23.157303+0000","last_became_active":"2024-09-11T09:28:52.722043+0000","last_became_peered":"2024-09-11T09:28:52.722043+0000","last_unstale":"2024-09-11T09:29:23.157303+0000","last_undegraded":"2024-09-11T09:29:23.157303+0000","last_fullsized":"2024-09-11T09:29:23.157303+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:09.415498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016227870000000001,"stat_sum":{"num_bytes":13114400,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2756,"num_read_kb":33135,"num_write":173,"num_write_kb":3786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":24576,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"49'424","reported_seq":10669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.221220+0000","last_change":"2024-09-11T09:28:48.704153+0000","last_active":"2024-09-11T09:29:23.221220+0000","last_peered":"2024-09-11T09:29:23.221220+0000","last_clean":"2024-09-11T09:29:23.221220+0000","last_became_active":"2024-09-11T09:28:46.704815+0000","last_became_peered":"2024-09-11T09:28:46.704815+0000","last_unstale":"2024-09-11T09:29:23.221220+0000","last_undegraded":"2024-09-11T09:29:23.221220+0000","last_fullsized":"2024-09-11T09:29:23.221220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:28.481867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077809799999999996,"stat_sum":{"num_bytes":5967872,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":11676,"num_read_kb":23264,"num_write":259,"num_write_kb":1118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.828904+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:29:22.828904+0000","last_peered":"2024-09-11T09:29:22.828904+0000","last_clean":"2024-09-11T09:29:22.828904+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:29:22.828904+0000","last_undegraded":"2024-09-11T09:29:22.828904+0000","last_fullsized":"2024-09-11T09:29:22.828904+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"49'905","reported_seq":6438,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.217039+0000","last_change":"2024-09-11T09:28:52.723617+0000","last_active":"2024-09-11T09:29:23.217039+0000","last_peered":"2024-09-11T09:29:23.217039+0000","last_clean":"2024-09-11T09:29:23.217039+0000","last_became_active":"2024-09-11T09:28:52.719138+0000","last_became_peered":"2024-09-11T09:28:52.719138+0000","last_unstale":"2024-09-11T09:29:23.217039+0000","last_undegraded":"2024-09-11T09:29:23.217039+0000","last_fullsized":"2024-09-11T09:29:23.217039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:54:30.815129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057083399999999999,"stat_sum":{"num_bytes":19676265,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":9869,"num_read_kb":33036,"num_write":1449,"num_write_kb":3577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":37888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816027+0000","last_change":"2024-09-11T09:28:52.717655+0000","last_active":"2024-09-11T09:29:22.816027+0000","last_peered":"2024-09-11T09:29:22.816027+0000","last_clean":"2024-09-11T09:29:22.816027+0000","last_became_active":"2024-09-11T09:28:52.717336+0000","last_became_peered":"2024-09-11T09:28:52.717336+0000","last_unstale":"2024-09-11T09:29:22.816027+0000","last_undegraded":"2024-09-11T09:29:22.816027+0000","last_fullsized":"2024-09-11T09:29:22.816027+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:45.628450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"47'171","reported_seq":6622,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.624475+0000","last_change":"2024-09-11T09:28:48.712120+0000","last_active":"2024-09-11T09:29:21.624475+0000","last_peered":"2024-09-11T09:29:21.624475+0000","last_clean":"2024-09-11T09:29:21.624475+0000","last_became_active":"2024-09-11T09:28:46.703147+0000","last_became_peered":"2024-09-11T09:28:46.703147+0000","last_unstale":"2024-09-11T09:29:21.624475+0000","last_undegraded":"2024-09-11T09:29:21.624475+0000","last_fullsized":"2024-09-11T09:29:21.624475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:04:16.604009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481087,"stat_sum":{"num_bytes":33046179,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":7284,"num_read_kb":45372,"num_write":135,"num_write_kb":1026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.815787+0000","last_change":"2024-09-11T09:28:38.615249+0000","last_active":"2024-09-11T09:29:22.815787+0000","last_peered":"2024-09-11T09:29:22.815787+0000","last_clean":"2024-09-11T09:29:22.815787+0000","last_became_active":"2024-09-11T09:28:36.453844+0000","last_became_peered":"2024-09-11T09:28:36.453844+0000","last_unstale":"2024-09-11T09:29:22.815787+0000","last_undegraded":"2024-09-11T09:29:22.815787+0000","last_fullsized":"2024-09-11T09:29:22.815787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:37.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047575299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"49'123","reported_seq":1369,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.117573+0000","last_change":"2024-09-11T09:28:48.699619+0000","last_active":"2024-09-11T09:29:23.117573+0000","last_peered":"2024-09-11T09:29:23.117573+0000","last_clean":"2024-09-11T09:29:23.117573+0000","last_became_active":"2024-09-11T09:28:46.698963+0000","last_became_peered":"2024-09-11T09:28:46.698963+0000","last_unstale":"2024-09-11T09:29:23.117573+0000","last_undegraded":"2024-09-11T09:29:23.117573+0000","last_fullsized":"2024-09-11T09:29:23.117573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:25.091373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055240599999999997,"stat_sum":{"num_bytes":17796640,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":1201,"num_read_kb":13391,"num_write":119,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.828928+0000","last_change":"2024-09-11T09:28:52.724806+0000","last_active":"2024-09-11T09:29:22.828928+0000","last_peered":"2024-09-11T09:29:22.828928+0000","last_clean":"2024-09-11T09:29:22.828928+0000","last_became_active":"2024-09-11T09:28:52.721789+0000","last_became_peered":"2024-09-11T09:28:52.721789+0000","last_unstale":"2024-09-11T09:29:22.828928+0000","last_undegraded":"2024-09-11T09:29:22.828928+0000","last_fullsized":"2024-09-11T09:29:22.828928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.928608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.815813+0000","last_change":"2024-09-11T09:28:30.806126+0000","last_active":"2024-09-11T09:29:22.815813+0000","last_peered":"2024-09-11T09:29:22.815813+0000","last_clean":"2024-09-11T09:29:22.815813+0000","last_became_active":"2024-09-11T09:28:30.805162+0000","last_became_peered":"2024-09-11T09:28:30.805162+0000","last_unstale":"2024-09-11T09:29:22.815813+0000","last_undegraded":"2024-09-11T09:29:22.815813+0000","last_fullsized":"2024-09-11T09:29:22.815813+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:57.740921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'1289","reported_seq":8008,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.217298+0000","last_change":"2024-09-11T09:28:52.723003+0000","last_active":"2024-09-11T09:29:23.217298+0000","last_peered":"2024-09-11T09:29:23.217298+0000","last_clean":"2024-09-11T09:29:23.217298+0000","last_became_active":"2024-09-11T09:28:52.722712+0000","last_became_peered":"2024-09-11T09:28:52.722712+0000","last_unstale":"2024-09-11T09:29:23.217298+0000","last_undegraded":"2024-09-11T09:29:23.217298+0000","last_fullsized":"2024-09-11T09:29:23.217298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1289,"log_dups_size":0,"ondisk_log_size":1289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:53.562480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40036190,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":13170,"num_read_kb":39166,"num_write":1922,"num_write_kb":3399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"49'260","reported_seq":1917,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.974708+0000","last_change":"2024-09-11T09:28:52.732916+0000","last_active":"2024-09-11T09:29:22.974708+0000","last_peered":"2024-09-11T09:29:22.974708+0000","last_clean":"2024-09-11T09:29:22.974708+0000","last_became_active":"2024-09-11T09:28:52.722767+0000","last_became_peered":"2024-09-11T09:28:52.722767+0000","last_unstale":"2024-09-11T09:29:22.974708+0000","last_undegraded":"2024-09-11T09:29:22.974708+0000","last_fullsized":"2024-09-11T09:29:22.974708+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:24.987132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042413100000000001,"stat_sum":{"num_bytes":3642451,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3613,"num_read_kb":3919,"num_write":221,"num_write_kb":862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":86016,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"49'492","reported_seq":4785,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.065424+0000","last_change":"2024-09-11T09:28:48.699539+0000","last_active":"2024-09-11T09:29:23.065424+0000","last_peered":"2024-09-11T09:29:23.065424+0000","last_clean":"2024-09-11T09:29:23.065424+0000","last_became_active":"2024-09-11T09:28:46.701226+0000","last_became_peered":"2024-09-11T09:28:46.701226+0000","last_unstale":"2024-09-11T09:29:23.065424+0000","last_undegraded":"2024-09-11T09:29:23.065424+0000","last_fullsized":"2024-09-11T09:29:23.065424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:56.089482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003117,"stat_sum":{"num_bytes":13028449,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":6033,"num_read_kb":22474,"num_write":768,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'133","reported_seq":2968,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.206393+0000","last_change":"2024-09-11T09:28:48.699375+0000","last_active":"2024-09-11T09:29:23.206393+0000","last_peered":"2024-09-11T09:29:23.206393+0000","last_clean":"2024-09-11T09:29:23.206393+0000","last_became_active":"2024-09-11T09:28:46.697192+0000","last_became_peered":"2024-09-11T09:28:46.697192+0000","last_unstale":"2024-09-11T09:29:23.206393+0000","last_undegraded":"2024-09-11T09:29:23.206393+0000","last_fullsized":"2024-09-11T09:29:23.206393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:15.145153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073583100000000005,"stat_sum":{"num_bytes":10608672,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2799,"num_read_kb":14974,"num_write":129,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"49'202","reported_seq":1436,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.976154+0000","last_change":"2024-09-11T09:28:48.708957+0000","last_active":"2024-09-11T09:29:22.976154+0000","last_peered":"2024-09-11T09:29:22.976154+0000","last_clean":"2024-09-11T09:29:22.976154+0000","last_became_active":"2024-09-11T09:28:46.701019+0000","last_became_peered":"2024-09-11T09:28:46.701019+0000","last_unstale":"2024-09-11T09:29:22.976154+0000","last_undegraded":"2024-09-11T09:29:22.976154+0000","last_fullsized":"2024-09-11T09:29:22.976154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:49.254360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413711,"stat_sum":{"num_bytes":1655808,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":1986,"num_read_kb":2887,"num_write":152,"num_write_kb":514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"49'207","reported_seq":2559,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.200097+0000","last_change":"2024-09-11T09:28:52.719245+0000","last_active":"2024-09-11T09:29:23.200097+0000","last_peered":"2024-09-11T09:29:23.200097+0000","last_clean":"2024-09-11T09:29:23.200097+0000","last_became_active":"2024-09-11T09:28:52.718920+0000","last_became_peered":"2024-09-11T09:28:52.718920+0000","last_unstale":"2024-09-11T09:29:23.200097+0000","last_undegraded":"2024-09-11T09:29:23.200097+0000","last_fullsized":"2024-09-11T09:29:23.200097+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:57:06.082009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086609299999999996,"stat_sum":{"num_bytes":15677952,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":2376,"num_read_kb":11380,"num_write":175,"num_write_kb":1745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"49'217","reported_seq":1796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.195856+0000","last_change":"2024-09-11T09:28:48.709547+0000","last_active":"2024-09-11T09:29:23.195856+0000","last_peered":"2024-09-11T09:29:23.195856+0000","last_clean":"2024-09-11T09:29:23.195856+0000","last_became_active":"2024-09-11T09:28:46.701376+0000","last_became_peered":"2024-09-11T09:28:46.701376+0000","last_unstale":"2024-09-11T09:29:23.195856+0000","last_undegraded":"2024-09-11T09:29:23.195856+0000","last_fullsized":"2024-09-11T09:29:23.195856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:22:41.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017071,"stat_sum":{"num_bytes":8831488,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":2844,"num_read_kb":5105,"num_write":184,"num_write_kb":2058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"49'177","reported_seq":5437,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.971251+0000","last_change":"2024-09-11T09:28:48.709797+0000","last_active":"2024-09-11T09:29:22.971251+0000","last_peered":"2024-09-11T09:29:22.971251+0000","last_clean":"2024-09-11T09:29:22.971251+0000","last_became_active":"2024-09-11T09:28:46.699421+0000","last_became_peered":"2024-09-11T09:28:46.699421+0000","last_unstale":"2024-09-11T09:29:22.971251+0000","last_undegraded":"2024-09-11T09:29:22.971251+0000","last_fullsized":"2024-09-11T09:29:22.971251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:51:06.622284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001864426,"stat_sum":{"num_bytes":39300112,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":5920,"num_read_kb":32103,"num_write":172,"num_write_kb":9853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"49'327","reported_seq":5287,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.205210+0000","last_change":"2024-09-11T09:28:48.707936+0000","last_active":"2024-09-11T09:29:23.205210+0000","last_peered":"2024-09-11T09:29:23.205210+0000","last_clean":"2024-09-11T09:29:23.205210+0000","last_became_active":"2024-09-11T09:28:46.701599+0000","last_became_peered":"2024-09-11T09:28:46.701599+0000","last_unstale":"2024-09-11T09:29:23.205210+0000","last_undegraded":"2024-09-11T09:29:23.205210+0000","last_fullsized":"2024-09-11T09:29:23.205210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:36.920823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049006899999999999,"stat_sum":{"num_bytes":5846054,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":5743,"num_read_kb":18349,"num_write":446,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"49'171","reported_seq":793,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.112574+0000","last_change":"2024-09-11T09:28:48.698936+0000","last_active":"2024-09-11T09:29:23.112574+0000","last_peered":"2024-09-11T09:29:23.112574+0000","last_clean":"2024-09-11T09:29:23.112574+0000","last_became_active":"2024-09-11T09:28:46.703566+0000","last_became_peered":"2024-09-11T09:28:46.703566+0000","last_unstale":"2024-09-11T09:29:23.112574+0000","last_undegraded":"2024-09-11T09:29:23.112574+0000","last_fullsized":"2024-09-11T09:29:23.112574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:32.878707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067916600000000004,"stat_sum":{"num_bytes":2640896,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":570,"num_read_kb":4797,"num_write":174,"num_write_kb":7770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"47'146","reported_seq":1790,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.555529+0000","last_change":"2024-09-11T09:28:48.713112+0000","last_active":"2024-09-11T09:29:21.555529+0000","last_peered":"2024-09-11T09:29:21.555529+0000","last_clean":"2024-09-11T09:29:21.555529+0000","last_became_active":"2024-09-11T09:28:46.700768+0000","last_became_peered":"2024-09-11T09:28:46.700768+0000","last_unstale":"2024-09-11T09:29:21.555529+0000","last_undegraded":"2024-09-11T09:29:21.555529+0000","last_fullsized":"2024-09-11T09:29:21.555529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:09.501328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054829999999999,"stat_sum":{"num_bytes":7218257,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":1656,"num_read_kb":9555,"num_write":136,"num_write_kb":3387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"49'206","reported_seq":3427,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.195809+0000","last_change":"2024-09-11T09:28:48.699348+0000","last_active":"2024-09-11T09:29:23.195809+0000","last_peered":"2024-09-11T09:29:23.195809+0000","last_clean":"2024-09-11T09:29:23.195809+0000","last_became_active":"2024-09-11T09:28:46.699767+0000","last_became_peered":"2024-09-11T09:28:46.699767+0000","last_unstale":"2024-09-11T09:29:23.195809+0000","last_undegraded":"2024-09-11T09:29:23.195809+0000","last_fullsized":"2024-09-11T09:29:23.195809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:59:01.489464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085759499999999995,"stat_sum":{"num_bytes":9672208,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":3817,"num_read_kb":6579,"num_write":172,"num_write_kb":1018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":387868856,"num_objects":807,"num_object_clones":226,"num_object_copies":1614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":807,"num_whiteouts":241,"num_read":121504,"num_read_kb":475320,"num_write":10469,"num_write_kb":108375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":148526,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16404480,"data_stored":114913136,"data_compressed":4525009,"data_compressed_allocated":8548352,"data_compressed_original":108323840,"omap_allocated":0,"internal_metadata":0},"log_size":8856,"ondisk_log_size":8856,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607566,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49032,"kb_used_data":3960,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322808,"statfs":{"total":96636764160,"available":96586555392,"internally_reserved":0,"allocated":4055040,"data_stored":27504893,"data_compressed":891267,"data_compressed_allocated":1871872,"data_compressed_original":25884672,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607566,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46984,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324856,"statfs":{"total":96636764160,"available":96588652544,"internally_reserved":0,"allocated":3395584,"data_stored":17921345,"data_compressed":936849,"data_compressed_allocated":1482752,"data_compressed_original":16650240,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43356,"kb_used_data":3852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328484,"statfs":{"total":96636764160,"available":96592367616,"internally_reserved":0,"allocated":3944448,"data_stored":25676638,"data_compressed":868885,"data_compressed_allocated":1908736,"data_compressed_original":24261120,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607567,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52716,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319124,"statfs":{"total":96636764160,"available":96582782976,"internally_reserved":0,"allocated":3559424,"data_stored":20037101,"data_compressed":946763,"data_compressed_allocated":1581056,"data_compressed_original":18553856,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640269,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42892,"kb_used_data":3384,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328948,"statfs":{"total":96636764160,"available":96592842752,"internally_reserved":0,"allocated":3465216,"data_stored":25265497,"data_compressed":898174,"data_compressed_allocated":1748992,"data_compressed_original":24075776,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3137536,"data_stored":24727684,"data_compressed":889783,"data_compressed_allocated":1728512,"data_compressed_original":23539200,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3108864,"data_stored":19914369,"data_compressed":946830,"data_compressed_allocated":1585152,"data_compressed_original":18562048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3481600,"data_stored":24974652,"data_compressed":860079,"data_compressed_allocated":1867776,"data_compressed_original":23663104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3035136,"data_stored":17870655,"data_compressed":936983,"data_compressed_allocated":1490944,"data_compressed_original":16666624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3641344,"data_stored":27425776,"data_compressed":891334,"data_compressed_allocated":1875968,"data_compressed_original":25892864,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:29:25.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:29:25.289 INFO:teuthology.orchestra.run.smithi107.stdout:2925 trunc from 0x456b400 to 0xbd86800 2024-09-11T09:29:25.312 INFO:teuthology.orchestra.run.smithi107.stdout:2926 write 0xc33f200 thru 0xc340133 (0xf34 bytes) 2024-09-11T09:29:25.318 INFO:teuthology.orchestra.run.smithi107.stdout:2933 write 0x6c5cc00 thru 0x6c6431d (0x771e bytes) 2024-09-11T09:29:25.318 INFO:teuthology.orchestra.run.smithi107.stdout:2936 read 0x311b000 thru 0x3124edd (0x9ede bytes) 2024-09-11T09:29:25.320 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:25.321+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:25.335 INFO:teuthology.orchestra.run.smithi107.stdout:2937 write 0xd317400 thru 0xd3210af (0x9cb0 bytes) 2024-09-11T09:29:25.341 INFO:teuthology.orchestra.run.smithi107.stdout:2938 read 0x479a000 thru 0x47a0c2b (0x6c2c bytes) 2024-09-11T09:29:25.349 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:25.349 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:25.355 INFO:teuthology.orchestra.run.smithi107.stdout:2939 trunc from 0xd321200 to 0x14df600 2024-09-11T09:29:25.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:25.378 INFO:teuthology.orchestra.run.smithi107.stdout:2940 read 0x1170400 thru 0x117f5b6 (0xf1b7 bytes) 2024-09-11T09:29:25.380 INFO:teuthology.orchestra.run.smithi107.stdout:2943 punch from 0x6e0800 to 0x6eba00, (0xb200 bytes) 2024-09-11T09:29:25.393 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:25.393+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:25.397 INFO:teuthology.orchestra.run.smithi107.stdout:2944 write 0x6e9ce00 thru 0x6ea5f18 (0x9119 bytes) 2024-09-11T09:29:25.397 INFO:teuthology.orchestra.run.smithi107.stdout:2946 read 0x2d38800 thru 0x2d441ea (0xb9eb bytes) 2024-09-11T09:29:25.420 INFO:teuthology.orchestra.run.smithi107.stdout:2952 write 0xb847000 thru 0xb856475 (0xf476 bytes) 2024-09-11T09:29:25.430 INFO:teuthology.orchestra.run.smithi107.stdout:2955 trunc from 0xb856600 to 0x7cd2400 2024-09-11T09:29:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:2956 read 0x6afd800 thru 0x6b012e4 (0x3ae5 bytes) 2024-09-11T09:29:25.462 INFO:teuthology.orchestra.run.smithi107.stdout:2957 read 0x35e1400 thru 0x35e25e1 (0x11e2 bytes) 2024-09-11T09:29:25.462 INFO:teuthology.orchestra.run.smithi107.stdout:2958 write 0x3a31400 thru 0x3a33616 (0x2217 bytes) 2024-09-11T09:29:25.462 INFO:teuthology.orchestra.run.smithi107.stdout:2960 write 0x3500800 thru 0x35067ad (0x5fae bytes) 2024-09-11T09:29:25.462 INFO:teuthology.orchestra.run.smithi107.stdout:2961 read 0x7931e00 thru 0x79388af (0x6ab0 bytes) 2024-09-11T09:29:25.463 INFO:teuthology.orchestra.run.smithi107.stdout:2966 trunc from 0x7cd2400 to 0x6472c00 2024-09-11T09:29:25.479 INFO:teuthology.orchestra.run.smithi107.stdout:2968 write 0x46dd800 thru 0x46e0b93 (0x3394 bytes) 2024-09-11T09:29:25.486 INFO:teuthology.orchestra.run.smithi107.stdout:2972 read 0x1d23a00 thru 0x1d248d4 (0xed5 bytes) 2024-09-11T09:29:25.487 INFO:teuthology.orchestra.run.smithi107.stdout:2973 read 0x141a400 thru 0x141c1b9 (0x1dba bytes) 2024-09-11T09:29:25.493 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:25.493+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:25.512 INFO:teuthology.orchestra.run.smithi107.stdout:2974 write 0xcdfb000 thru 0xcdfda91 (0x2a92 bytes) 2024-09-11T09:29:25.516 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:25.517 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:25.517 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:25.517 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:25.517 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:25.518 INFO:teuthology.orchestra.run.smithi107.stdout:2975 write 0x8017e00 thru 0x80184e6 (0x6e7 bytes) 2024-09-11T09:29:25.520 INFO:teuthology.orchestra.run.smithi107.stdout:2977 trunc from 0xcdfdc00 to 0xbd06800 2024-09-11T09:29:25.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:25.546 INFO:teuthology.orchestra.run.smithi107.stdout:2979 read 0x7039c00 thru 0x703da9b (0x3e9c bytes) 2024-09-11T09:29:25.568 INFO:teuthology.orchestra.run.smithi107.stdout:2984 write 0xcf72200 thru 0xcf76f92 (0x4d93 bytes) 2024-09-11T09:29:25.568 INFO:teuthology.orchestra.run.smithi107.stdout:2985 read 0x3222000 thru 0x32268c0 (0x48c1 bytes) 2024-09-11T09:29:25.568 INFO:teuthology.orchestra.run.smithi107.stdout:2986 write 0x34a5800 thru 0x34aff48 (0xa749 bytes) 2024-09-11T09:29:25.568 INFO:teuthology.orchestra.run.smithi107.stdout:2989 write 0x874dc00 thru 0x875359e (0x599f bytes) 2024-09-11T09:29:25.568 INFO:teuthology.orchestra.run.smithi107.stdout:2992 punch from 0x739d000 to 0x739d600, (0x600 bytes) 2024-09-11T09:29:25.570 INFO:teuthology.orchestra.run.smithi107.stdout:2994 read 0x29f7000 thru 0x2a03e68 (0xce69 bytes) 2024-09-11T09:29:25.570 INFO:teuthology.orchestra.run.smithi107.stdout:2995 read 0x3e3d000 thru 0x3e4a04f (0xd050 bytes) 2024-09-11T09:29:25.570 INFO:teuthology.orchestra.run.smithi107.stdout:2997 write 0x394d000 thru 0x395b421 (0xe422 bytes) 2024-09-11T09:29:25.570 INFO:teuthology.orchestra.run.smithi107.stdout:3002 read 0x3119c00 thru 0x311a2de (0x6df bytes) 2024-09-11T09:29:25.570 INFO:teuthology.orchestra.run.smithi107.stdout:3004 punch from 0x9c8ac00 to 0x9c8ec00, (0x4000 bytes) 2024-09-11T09:29:25.570 INFO:teuthology.orchestra.run.smithi107.stdout:3011 read 0xbd5e200 thru 0xbd6afa8 (0xcda9 bytes) 2024-09-11T09:29:25.572 INFO:teuthology.orchestra.run.smithi107.stdout:3018 trunc from 0xcf77000 to 0xe66bc00 2024-09-11T09:29:25.594 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:25.594+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:25.597 INFO:teuthology.orchestra.run.smithi107.stdout:3020 read 0x1507400 thru 0x1511266 (0x9e67 bytes) 2024-09-11T09:29:25.598 INFO:teuthology.orchestra.run.smithi107.stdout:3023 read 0x4b5a00 thru 0x4b79b3 (0x1fb4 bytes) 2024-09-11T09:29:25.598 INFO:teuthology.orchestra.run.smithi107.stdout:3024 write 0xe02ce00 thru 0xe0346f2 (0x78f3 bytes) 2024-09-11T09:29:25.598 INFO:teuthology.orchestra.run.smithi107.stdout:3025 punch from 0x400e000 to 0x400e400, (0x400 bytes) 2024-09-11T09:29:25.599 INFO:teuthology.orchestra.run.smithi107.stdout:3026 write 0x60b000 thru 0x614242 (0x9243 bytes) 2024-09-11T09:29:25.607 INFO:teuthology.orchestra.run.smithi107.stdout:3027 trunc from 0xe66bc00 to 0x7c47400 2024-09-11T09:29:25.626 INFO:teuthology.orchestra.run.smithi107.stdout:3029 write 0x34d3400 thru 0x34d7adb (0x46dc bytes) 2024-09-11T09:29:25.627 INFO:teuthology.orchestra.run.smithi107.stdout:3031 write 0x58da400 thru 0x58decf2 (0x48f3 bytes) 2024-09-11T09:29:25.627 INFO:teuthology.orchestra.run.smithi107.stdout:3032 read 0x4f55e00 thru 0x4f5fe23 (0xa024 bytes) 2024-09-11T09:29:25.627 INFO:teuthology.orchestra.run.smithi107.stdout:3034 read 0x17d7800 thru 0x17dcab2 (0x52b3 bytes) 2024-09-11T09:29:25.628 INFO:teuthology.orchestra.run.smithi107.stdout:3035 trunc from 0x7c47400 to 0x7d94c00 2024-09-11T09:29:25.635 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:29:25.636 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:29:25.649 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-11T09:29:24.824661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":388459243,"num_objects":811,"num_object_clones":226,"num_object_copies":1622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":811,"num_whiteouts":241,"num_read":121620,"num_read_kb":475420,"num_write":10630,"num_write_kb":110793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":148526,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8976,"ondisk_log_size":8976,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":234980,"kb_used_data":17988,"kb_used_omap":7,"kb_used_meta":216888,"kb_avail":471624220,"statfs":{"total":483183820800,"available":482943201280,"internally_reserved":0,"allocated":18419712,"data_stored":116405474,"data_compressed":4541938,"data_compressed_allocated":8593408,"data_compressed_original":109425664,"omap_allocated":7973,"internal_metadata":222093531},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20144141,"num_objects":172,"num_object_clones":26,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":21,"num_read":23024,"num_read_kb":63394,"num_write":3795,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017373"},"pg_stats":[{"pgid":"3.1f","version":"47'169","reported_seq":1304,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.618965+0000","last_change":"2024-09-11T09:28:48.712900+0000","last_active":"2024-09-11T09:29:21.618965+0000","last_peered":"2024-09-11T09:29:21.618965+0000","last_clean":"2024-09-11T09:29:21.618965+0000","last_became_active":"2024-09-11T09:28:46.699073+0000","last_became_peered":"2024-09-11T09:28:46.699073+0000","last_unstale":"2024-09-11T09:29:21.618965+0000","last_undegraded":"2024-09-11T09:29:21.618965+0000","last_fullsized":"2024-09-11T09:29:21.618965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:03.561937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012367610000000001,"stat_sum":{"num_bytes":17482768,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":1112,"num_read_kb":9691,"num_write":173,"num_write_kb":9430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"49'307","reported_seq":2811,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.153098+0000","last_change":"2024-09-11T09:28:52.719933+0000","last_active":"2024-09-11T09:29:23.153098+0000","last_peered":"2024-09-11T09:29:23.153098+0000","last_clean":"2024-09-11T09:29:23.153098+0000","last_became_active":"2024-09-11T09:28:52.715808+0000","last_became_peered":"2024-09-11T09:28:52.715808+0000","last_unstale":"2024-09-11T09:29:23.153098+0000","last_undegraded":"2024-09-11T09:29:23.153098+0000","last_fullsized":"2024-09-11T09:29:23.153098+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:27:42.284486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001676,"stat_sum":{"num_bytes":12437337,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3658,"num_read_kb":18869,"num_write":303,"num_write_kb":8025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"49'390","reported_seq":1441,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.829014+0000","last_change":"2024-09-11T09:28:48.709187+0000","last_active":"2024-09-11T09:29:22.829014+0000","last_peered":"2024-09-11T09:29:22.829014+0000","last_clean":"2024-09-11T09:29:22.829014+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:29:22.829014+0000","last_undegraded":"2024-09-11T09:29:22.829014+0000","last_fullsized":"2024-09-11T09:29:22.829014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:46.255889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015577869999999999,"stat_sum":{"num_bytes":22194763,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2441,"num_read_kb":19626,"num_write":606,"num_write_kb":9462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"49'154","reported_seq":5357,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.975838+0000","last_change":"2024-09-11T09:28:48.708858+0000","last_active":"2024-09-11T09:29:22.975838+0000","last_peered":"2024-09-11T09:29:22.975838+0000","last_clean":"2024-09-11T09:29:22.975838+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:29:22.975838+0000","last_undegraded":"2024-09-11T09:29:22.975838+0000","last_fullsized":"2024-09-11T09:29:22.975838+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:17.234093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013196460000000001,"stat_sum":{"num_bytes":11452456,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":9,"num_read":5210,"num_read_kb":7606,"num_write":144,"num_write_kb":2075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"47'120","reported_seq":1526,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.728268+0000","last_change":"2024-09-11T09:28:48.712945+0000","last_active":"2024-09-11T09:29:21.728268+0000","last_peered":"2024-09-11T09:29:21.728268+0000","last_clean":"2024-09-11T09:29:21.728268+0000","last_became_active":"2024-09-11T09:28:46.702512+0000","last_became_peered":"2024-09-11T09:28:46.702512+0000","last_unstale":"2024-09-11T09:29:21.728268+0000","last_undegraded":"2024-09-11T09:29:21.728268+0000","last_fullsized":"2024-09-11T09:29:21.728268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:37:42.492275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012800909999999999,"stat_sum":{"num_bytes":3618304,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":1358,"num_read_kb":8030,"num_write":115,"num_write_kb":1455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"48'232","reported_seq":3060,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.166706+0000","last_change":"2024-09-11T09:28:48.704277+0000","last_active":"2024-09-11T09:29:23.166706+0000","last_peered":"2024-09-11T09:29:23.166706+0000","last_clean":"2024-09-11T09:29:23.166706+0000","last_became_active":"2024-09-11T09:28:46.704959+0000","last_became_peered":"2024-09-11T09:28:46.704959+0000","last_unstale":"2024-09-11T09:29:23.166706+0000","last_undegraded":"2024-09-11T09:29:23.166706+0000","last_fullsized":"2024-09-11T09:29:23.166706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:13:26.702896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081867,"stat_sum":{"num_bytes":3345531,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":3268,"num_read_kb":11603,"num_write":287,"num_write_kb":962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"49'194","reported_seq":551,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.972664+0000","last_change":"2024-09-11T09:28:48.699507+0000","last_active":"2024-09-11T09:29:22.972664+0000","last_peered":"2024-09-11T09:29:22.972664+0000","last_clean":"2024-09-11T09:29:22.972664+0000","last_became_active":"2024-09-11T09:28:46.701636+0000","last_became_peered":"2024-09-11T09:28:46.701636+0000","last_unstale":"2024-09-11T09:29:22.972664+0000","last_undegraded":"2024-09-11T09:29:22.972664+0000","last_fullsized":"2024-09-11T09:29:22.972664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:45:18.829520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058014000000000004,"stat_sum":{"num_bytes":3833474,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":466,"num_read_kb":2749,"num_write":211,"num_write_kb":3202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"47'215","reported_seq":904,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.655623+0000","last_change":"2024-09-11T09:28:52.717462+0000","last_active":"2024-09-11T09:29:21.655623+0000","last_peered":"2024-09-11T09:29:21.655623+0000","last_clean":"2024-09-11T09:29:21.655623+0000","last_became_active":"2024-09-11T09:28:52.717165+0000","last_became_peered":"2024-09-11T09:28:52.717165+0000","last_unstale":"2024-09-11T09:29:21.655623+0000","last_undegraded":"2024-09-11T09:29:21.655623+0000","last_fullsized":"2024-09-11T09:29:21.655623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:59.930098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5226532,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":1206,"num_read_kb":7262,"num_write":288,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"47'203","reported_seq":3411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.542390+0000","last_change":"2024-09-11T09:28:48.713192+0000","last_active":"2024-09-11T09:29:21.542390+0000","last_peered":"2024-09-11T09:29:21.542390+0000","last_clean":"2024-09-11T09:29:21.542390+0000","last_became_active":"2024-09-11T09:28:46.702717+0000","last_became_peered":"2024-09-11T09:28:46.702717+0000","last_unstale":"2024-09-11T09:29:21.542390+0000","last_undegraded":"2024-09-11T09:29:21.542390+0000","last_fullsized":"2024-09-11T09:29:21.542390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:45:41.787524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001366531,"stat_sum":{"num_bytes":8125637,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":3594,"num_read_kb":16530,"num_write":263,"num_write_kb":8540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"49'545","reported_seq":5763,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.207538+0000","last_change":"2024-09-11T09:28:48.709601+0000","last_active":"2024-09-11T09:29:23.207538+0000","last_peered":"2024-09-11T09:29:23.207538+0000","last_clean":"2024-09-11T09:29:23.207538+0000","last_became_active":"2024-09-11T09:28:46.699041+0000","last_became_peered":"2024-09-11T09:28:46.699041+0000","last_unstale":"2024-09-11T09:29:23.207538+0000","last_undegraded":"2024-09-11T09:29:23.207538+0000","last_fullsized":"2024-09-11T09:29:23.207538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:57:27.083097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017410150000000001,"stat_sum":{"num_bytes":7954049,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":9083,"num_read_kb":23429,"num_write":642,"num_write_kb":2017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"49'151","reported_seq":1515,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.118722+0000","last_change":"2024-09-11T09:28:48.699698+0000","last_active":"2024-09-11T09:29:23.118722+0000","last_peered":"2024-09-11T09:29:23.118722+0000","last_clean":"2024-09-11T09:29:23.118722+0000","last_became_active":"2024-09-11T09:28:46.700365+0000","last_became_peered":"2024-09-11T09:28:46.700365+0000","last_unstale":"2024-09-11T09:29:23.118722+0000","last_undegraded":"2024-09-11T09:29:23.118722+0000","last_fullsized":"2024-09-11T09:29:23.118722+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:03.735174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014058110000000001,"stat_sum":{"num_bytes":9364049,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":9,"num_read":1346,"num_read_kb":3414,"num_write":142,"num_write_kb":866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"49'124","reported_seq":375,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.064909+0000","last_change":"2024-09-11T09:28:48.698947+0000","last_active":"2024-09-11T09:29:23.064909+0000","last_peered":"2024-09-11T09:29:23.064909+0000","last_clean":"2024-09-11T09:29:23.064909+0000","last_became_active":"2024-09-11T09:28:46.703111+0000","last_became_peered":"2024-09-11T09:28:46.703111+0000","last_unstale":"2024-09-11T09:29:23.064909+0000","last_undegraded":"2024-09-11T09:29:23.064909+0000","last_fullsized":"2024-09-11T09:29:23.064909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:43:20.769364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063234499999999998,"stat_sum":{"num_bytes":2714639,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":189,"num_read_kb":2133,"num_write":121,"num_write_kb":727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816167+0000","last_change":"2024-09-11T09:28:38.615508+0000","last_active":"2024-09-11T09:29:22.816167+0000","last_peered":"2024-09-11T09:29:22.816167+0000","last_clean":"2024-09-11T09:29:22.816167+0000","last_became_active":"2024-09-11T09:28:36.535007+0000","last_became_peered":"2024-09-11T09:28:36.535007+0000","last_unstale":"2024-09-11T09:29:22.816167+0000","last_undegraded":"2024-09-11T09:29:22.816167+0000","last_fullsized":"2024-09-11T09:29:22.816167+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:46.010169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068309000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'124","reported_seq":1442,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.716770+0000","last_change":"2024-09-11T09:28:48.712155+0000","last_active":"2024-09-11T09:29:21.716770+0000","last_peered":"2024-09-11T09:29:21.716770+0000","last_clean":"2024-09-11T09:29:21.716770+0000","last_became_active":"2024-09-11T09:28:46.700016+0000","last_became_peered":"2024-09-11T09:28:46.700016+0000","last_unstale":"2024-09-11T09:29:21.716770+0000","last_undegraded":"2024-09-11T09:29:21.716770+0000","last_fullsized":"2024-09-11T09:29:21.716770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:38:46.091365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051448599999999998,"stat_sum":{"num_bytes":14400000,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":1267,"num_read_kb":5262,"num_write":117,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816192+0000","last_change":"2024-09-11T09:28:38.462223+0000","last_active":"2024-09-11T09:29:22.816192+0000","last_peered":"2024-09-11T09:29:22.816192+0000","last_clean":"2024-09-11T09:29:22.816192+0000","last_became_active":"2024-09-11T09:28:36.455841+0000","last_became_peered":"2024-09-11T09:28:36.455841+0000","last_unstale":"2024-09-11T09:29:22.816192+0000","last_undegraded":"2024-09-11T09:29:22.816192+0000","last_fullsized":"2024-09-11T09:29:22.816192+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:26.492117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"49'121","reported_seq":3518,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.037039+0000","last_change":"2024-09-11T09:28:52.719363+0000","last_active":"2024-09-11T09:29:23.037039+0000","last_peered":"2024-09-11T09:29:23.037039+0000","last_clean":"2024-09-11T09:29:23.037039+0000","last_became_active":"2024-09-11T09:28:52.719015+0000","last_became_peered":"2024-09-11T09:28:52.719015+0000","last_unstale":"2024-09-11T09:29:23.037039+0000","last_undegraded":"2024-09-11T09:29:23.037039+0000","last_fullsized":"2024-09-11T09:29:23.037039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:46.018964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7513600,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":3355,"num_read_kb":9852,"num_write":116,"num_write_kb":2383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816107+0000","last_change":"2024-09-11T09:28:38.615243+0000","last_active":"2024-09-11T09:29:22.816107+0000","last_peered":"2024-09-11T09:29:22.816107+0000","last_clean":"2024-09-11T09:29:22.816107+0000","last_became_active":"2024-09-11T09:28:36.453319+0000","last_became_peered":"2024-09-11T09:28:36.453319+0000","last_unstale":"2024-09-11T09:29:22.816107+0000","last_undegraded":"2024-09-11T09:29:22.816107+0000","last_fullsized":"2024-09-11T09:29:22.816107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:41.513882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047127699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"49'161","reported_seq":842,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.007384+0000","last_change":"2024-09-11T09:28:52.718252+0000","last_active":"2024-09-11T09:29:23.007384+0000","last_peered":"2024-09-11T09:29:23.007384+0000","last_clean":"2024-09-11T09:29:23.007384+0000","last_became_active":"2024-09-11T09:28:52.717825+0000","last_became_peered":"2024-09-11T09:28:52.717825+0000","last_unstale":"2024-09-11T09:29:23.007384+0000","last_undegraded":"2024-09-11T09:29:23.007384+0000","last_fullsized":"2024-09-11T09:29:23.007384+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:18:32.603140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729355,"stat_sum":{"num_bytes":10445824,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":638,"num_read_kb":9778,"num_write":155,"num_write_kb":3807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816127+0000","last_change":"2024-09-11T09:28:38.462264+0000","last_active":"2024-09-11T09:29:22.816127+0000","last_peered":"2024-09-11T09:29:22.816127+0000","last_clean":"2024-09-11T09:29:22.816127+0000","last_became_active":"2024-09-11T09:28:36.454444+0000","last_became_peered":"2024-09-11T09:28:36.454444+0000","last_unstale":"2024-09-11T09:29:22.816127+0000","last_undegraded":"2024-09-11T09:29:22.816127+0000","last_fullsized":"2024-09-11T09:29:22.816127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:56.449791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050275600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"49'196","reported_seq":2724,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.157303+0000","last_change":"2024-09-11T09:28:52.729839+0000","last_active":"2024-09-11T09:29:23.157303+0000","last_peered":"2024-09-11T09:29:23.157303+0000","last_clean":"2024-09-11T09:29:23.157303+0000","last_became_active":"2024-09-11T09:28:52.722043+0000","last_became_peered":"2024-09-11T09:28:52.722043+0000","last_unstale":"2024-09-11T09:29:23.157303+0000","last_undegraded":"2024-09-11T09:29:23.157303+0000","last_fullsized":"2024-09-11T09:29:23.157303+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:01:09.415498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016227870000000001,"stat_sum":{"num_bytes":13114400,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2756,"num_read_kb":33135,"num_write":173,"num_write_kb":3786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":24576,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"49'424","reported_seq":10669,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.221220+0000","last_change":"2024-09-11T09:28:48.704153+0000","last_active":"2024-09-11T09:29:23.221220+0000","last_peered":"2024-09-11T09:29:23.221220+0000","last_clean":"2024-09-11T09:29:23.221220+0000","last_became_active":"2024-09-11T09:28:46.704815+0000","last_became_peered":"2024-09-11T09:28:46.704815+0000","last_unstale":"2024-09-11T09:29:23.221220+0000","last_undegraded":"2024-09-11T09:29:23.221220+0000","last_fullsized":"2024-09-11T09:29:23.221220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:28.481867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077809799999999996,"stat_sum":{"num_bytes":5967872,"num_objects":34,"num_object_clones":12,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":11676,"num_read_kb":23264,"num_write":259,"num_write_kb":1118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.828904+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:29:22.828904+0000","last_peered":"2024-09-11T09:29:22.828904+0000","last_clean":"2024-09-11T09:29:22.828904+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:29:22.828904+0000","last_undegraded":"2024-09-11T09:29:22.828904+0000","last_fullsized":"2024-09-11T09:29:22.828904+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"49'905","reported_seq":6438,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.217039+0000","last_change":"2024-09-11T09:28:52.723617+0000","last_active":"2024-09-11T09:29:23.217039+0000","last_peered":"2024-09-11T09:29:23.217039+0000","last_clean":"2024-09-11T09:29:23.217039+0000","last_became_active":"2024-09-11T09:28:52.719138+0000","last_became_peered":"2024-09-11T09:28:52.719138+0000","last_unstale":"2024-09-11T09:29:23.217039+0000","last_undegraded":"2024-09-11T09:29:23.217039+0000","last_fullsized":"2024-09-11T09:29:23.217039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:54:30.815129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057083399999999999,"stat_sum":{"num_bytes":19676265,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":7,"num_read":9869,"num_read_kb":33036,"num_write":1449,"num_write_kb":3577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":37888,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.816027+0000","last_change":"2024-09-11T09:28:52.717655+0000","last_active":"2024-09-11T09:29:22.816027+0000","last_peered":"2024-09-11T09:29:22.816027+0000","last_clean":"2024-09-11T09:29:22.816027+0000","last_became_active":"2024-09-11T09:28:52.717336+0000","last_became_peered":"2024-09-11T09:28:52.717336+0000","last_unstale":"2024-09-11T09:29:22.816027+0000","last_undegraded":"2024-09-11T09:29:22.816027+0000","last_fullsized":"2024-09-11T09:29:22.816027+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:45.628450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"47'171","reported_seq":6622,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.624475+0000","last_change":"2024-09-11T09:28:48.712120+0000","last_active":"2024-09-11T09:29:21.624475+0000","last_peered":"2024-09-11T09:29:21.624475+0000","last_clean":"2024-09-11T09:29:21.624475+0000","last_became_active":"2024-09-11T09:28:46.703147+0000","last_became_peered":"2024-09-11T09:28:46.703147+0000","last_unstale":"2024-09-11T09:29:21.624475+0000","last_undegraded":"2024-09-11T09:29:21.624475+0000","last_fullsized":"2024-09-11T09:29:21.624475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:04:16.604009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481087,"stat_sum":{"num_bytes":33046179,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":7284,"num_read_kb":45372,"num_write":135,"num_write_kb":1026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.815787+0000","last_change":"2024-09-11T09:28:38.615249+0000","last_active":"2024-09-11T09:29:22.815787+0000","last_peered":"2024-09-11T09:29:22.815787+0000","last_clean":"2024-09-11T09:29:22.815787+0000","last_became_active":"2024-09-11T09:28:36.453844+0000","last_became_peered":"2024-09-11T09:28:36.453844+0000","last_unstale":"2024-09-11T09:29:22.815787+0000","last_undegraded":"2024-09-11T09:29:22.815787+0000","last_fullsized":"2024-09-11T09:29:22.815787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:36:37.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047575299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"49'123","reported_seq":1369,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.117573+0000","last_change":"2024-09-11T09:28:48.699619+0000","last_active":"2024-09-11T09:29:23.117573+0000","last_peered":"2024-09-11T09:29:23.117573+0000","last_clean":"2024-09-11T09:29:23.117573+0000","last_became_active":"2024-09-11T09:28:46.698963+0000","last_became_peered":"2024-09-11T09:28:46.698963+0000","last_unstale":"2024-09-11T09:29:23.117573+0000","last_undegraded":"2024-09-11T09:29:23.117573+0000","last_fullsized":"2024-09-11T09:29:23.117573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:25.091373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055240599999999997,"stat_sum":{"num_bytes":17796640,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":1201,"num_read_kb":13391,"num_write":119,"num_write_kb":924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.828928+0000","last_change":"2024-09-11T09:28:52.724806+0000","last_active":"2024-09-11T09:29:22.828928+0000","last_peered":"2024-09-11T09:29:22.828928+0000","last_clean":"2024-09-11T09:29:22.828928+0000","last_became_active":"2024-09-11T09:28:52.721789+0000","last_became_peered":"2024-09-11T09:28:52.721789+0000","last_unstale":"2024-09-11T09:29:22.828928+0000","last_undegraded":"2024-09-11T09:29:22.828928+0000","last_fullsized":"2024-09-11T09:29:22.828928+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.928608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":230,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.815813+0000","last_change":"2024-09-11T09:28:30.806126+0000","last_active":"2024-09-11T09:29:22.815813+0000","last_peered":"2024-09-11T09:29:22.815813+0000","last_clean":"2024-09-11T09:29:22.815813+0000","last_became_active":"2024-09-11T09:28:30.805162+0000","last_became_peered":"2024-09-11T09:28:30.805162+0000","last_unstale":"2024-09-11T09:29:22.815813+0000","last_undegraded":"2024-09-11T09:29:22.815813+0000","last_fullsized":"2024-09-11T09:29:22.815813+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:57.740921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"49'1289","reported_seq":8008,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.217298+0000","last_change":"2024-09-11T09:28:52.723003+0000","last_active":"2024-09-11T09:29:23.217298+0000","last_peered":"2024-09-11T09:29:23.217298+0000","last_clean":"2024-09-11T09:29:23.217298+0000","last_became_active":"2024-09-11T09:28:52.722712+0000","last_became_peered":"2024-09-11T09:28:52.722712+0000","last_unstale":"2024-09-11T09:29:23.217298+0000","last_undegraded":"2024-09-11T09:29:23.217298+0000","last_fullsized":"2024-09-11T09:29:23.217298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1289,"log_dups_size":0,"ondisk_log_size":1289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:53.562480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40036190,"num_objects":46,"num_object_clones":15,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":13170,"num_read_kb":39166,"num_write":1922,"num_write_kb":3399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"49'260","reported_seq":1917,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.974708+0000","last_change":"2024-09-11T09:28:52.732916+0000","last_active":"2024-09-11T09:29:22.974708+0000","last_peered":"2024-09-11T09:29:22.974708+0000","last_clean":"2024-09-11T09:29:22.974708+0000","last_became_active":"2024-09-11T09:28:52.722767+0000","last_became_peered":"2024-09-11T09:28:52.722767+0000","last_unstale":"2024-09-11T09:29:22.974708+0000","last_undegraded":"2024-09-11T09:29:22.974708+0000","last_fullsized":"2024-09-11T09:29:22.974708+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":260,"log_dups_size":0,"ondisk_log_size":260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:24.987132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042413100000000001,"stat_sum":{"num_bytes":3642451,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3613,"num_read_kb":3919,"num_write":221,"num_write_kb":862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":86016,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"49'492","reported_seq":4785,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.065424+0000","last_change":"2024-09-11T09:28:48.699539+0000","last_active":"2024-09-11T09:29:23.065424+0000","last_peered":"2024-09-11T09:29:23.065424+0000","last_clean":"2024-09-11T09:29:23.065424+0000","last_became_active":"2024-09-11T09:28:46.701226+0000","last_became_peered":"2024-09-11T09:28:46.701226+0000","last_unstale":"2024-09-11T09:29:23.065424+0000","last_undegraded":"2024-09-11T09:29:23.065424+0000","last_fullsized":"2024-09-11T09:29:23.065424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:30:56.089482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003117,"stat_sum":{"num_bytes":13028449,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":6033,"num_read_kb":22474,"num_write":768,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"49'133","reported_seq":2968,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.206393+0000","last_change":"2024-09-11T09:28:48.699375+0000","last_active":"2024-09-11T09:29:23.206393+0000","last_peered":"2024-09-11T09:29:23.206393+0000","last_clean":"2024-09-11T09:29:23.206393+0000","last_became_active":"2024-09-11T09:28:46.697192+0000","last_became_peered":"2024-09-11T09:28:46.697192+0000","last_unstale":"2024-09-11T09:29:23.206393+0000","last_undegraded":"2024-09-11T09:29:23.206393+0000","last_fullsized":"2024-09-11T09:29:23.206393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:15.145153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073583100000000005,"stat_sum":{"num_bytes":10608672,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":2799,"num_read_kb":14974,"num_write":129,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"49'202","reported_seq":1436,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.976154+0000","last_change":"2024-09-11T09:28:48.708957+0000","last_active":"2024-09-11T09:29:22.976154+0000","last_peered":"2024-09-11T09:29:22.976154+0000","last_clean":"2024-09-11T09:29:22.976154+0000","last_became_active":"2024-09-11T09:28:46.701019+0000","last_became_peered":"2024-09-11T09:28:46.701019+0000","last_unstale":"2024-09-11T09:29:22.976154+0000","last_undegraded":"2024-09-11T09:29:22.976154+0000","last_fullsized":"2024-09-11T09:29:22.976154+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:49.254360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413711,"stat_sum":{"num_bytes":1655808,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":9,"num_read":1986,"num_read_kb":2887,"num_write":152,"num_write_kb":514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"49'207","reported_seq":2559,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.200097+0000","last_change":"2024-09-11T09:28:52.719245+0000","last_active":"2024-09-11T09:29:23.200097+0000","last_peered":"2024-09-11T09:29:23.200097+0000","last_clean":"2024-09-11T09:29:23.200097+0000","last_became_active":"2024-09-11T09:28:52.718920+0000","last_became_peered":"2024-09-11T09:28:52.718920+0000","last_unstale":"2024-09-11T09:29:23.200097+0000","last_undegraded":"2024-09-11T09:29:23.200097+0000","last_fullsized":"2024-09-11T09:29:23.200097+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:57:06.082009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086609299999999996,"stat_sum":{"num_bytes":15677952,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":2376,"num_read_kb":11380,"num_write":175,"num_write_kb":1745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"49'217","reported_seq":1796,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.195856+0000","last_change":"2024-09-11T09:28:48.709547+0000","last_active":"2024-09-11T09:29:23.195856+0000","last_peered":"2024-09-11T09:29:23.195856+0000","last_clean":"2024-09-11T09:29:23.195856+0000","last_became_active":"2024-09-11T09:28:46.701376+0000","last_became_peered":"2024-09-11T09:28:46.701376+0000","last_unstale":"2024-09-11T09:29:23.195856+0000","last_undegraded":"2024-09-11T09:29:23.195856+0000","last_fullsized":"2024-09-11T09:29:23.195856+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:22:41.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017071,"stat_sum":{"num_bytes":8831488,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":2844,"num_read_kb":5105,"num_write":184,"num_write_kb":2058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"49'177","reported_seq":5437,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:22.971251+0000","last_change":"2024-09-11T09:28:48.709797+0000","last_active":"2024-09-11T09:29:22.971251+0000","last_peered":"2024-09-11T09:29:22.971251+0000","last_clean":"2024-09-11T09:29:22.971251+0000","last_became_active":"2024-09-11T09:28:46.699421+0000","last_became_peered":"2024-09-11T09:28:46.699421+0000","last_unstale":"2024-09-11T09:29:22.971251+0000","last_undegraded":"2024-09-11T09:29:22.971251+0000","last_fullsized":"2024-09-11T09:29:22.971251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:51:06.622284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001864426,"stat_sum":{"num_bytes":39300112,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":5920,"num_read_kb":32103,"num_write":172,"num_write_kb":9853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"49'327","reported_seq":5287,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.205210+0000","last_change":"2024-09-11T09:28:48.707936+0000","last_active":"2024-09-11T09:29:23.205210+0000","last_peered":"2024-09-11T09:29:23.205210+0000","last_clean":"2024-09-11T09:29:23.205210+0000","last_became_active":"2024-09-11T09:28:46.701599+0000","last_became_peered":"2024-09-11T09:28:46.701599+0000","last_unstale":"2024-09-11T09:29:23.205210+0000","last_undegraded":"2024-09-11T09:29:23.205210+0000","last_fullsized":"2024-09-11T09:29:23.205210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:36.920823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049006899999999999,"stat_sum":{"num_bytes":5846054,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":5743,"num_read_kb":18349,"num_write":446,"num_write_kb":3939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"49'171","reported_seq":793,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.112574+0000","last_change":"2024-09-11T09:28:48.698936+0000","last_active":"2024-09-11T09:29:23.112574+0000","last_peered":"2024-09-11T09:29:23.112574+0000","last_clean":"2024-09-11T09:29:23.112574+0000","last_became_active":"2024-09-11T09:28:46.703566+0000","last_became_peered":"2024-09-11T09:28:46.703566+0000","last_unstale":"2024-09-11T09:29:23.112574+0000","last_undegraded":"2024-09-11T09:29:23.112574+0000","last_fullsized":"2024-09-11T09:29:23.112574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:32.878707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067916600000000004,"stat_sum":{"num_bytes":2640896,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":570,"num_read_kb":4797,"num_write":174,"num_write_kb":7770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"47'146","reported_seq":1790,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T09:29:21.555529+0000","last_change":"2024-09-11T09:28:48.713112+0000","last_active":"2024-09-11T09:29:21.555529+0000","last_peered":"2024-09-11T09:29:21.555529+0000","last_clean":"2024-09-11T09:29:21.555529+0000","last_became_active":"2024-09-11T09:28:46.700768+0000","last_became_peered":"2024-09-11T09:28:46.700768+0000","last_unstale":"2024-09-11T09:29:21.555529+0000","last_undegraded":"2024-09-11T09:29:21.555529+0000","last_fullsized":"2024-09-11T09:29:21.555529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:09.501328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054829999999999,"stat_sum":{"num_bytes":7218257,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":11,"num_read":1656,"num_read_kb":9555,"num_write":136,"num_write_kb":3387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"49'206","reported_seq":3427,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T09:29:23.195809+0000","last_change":"2024-09-11T09:28:48.699348+0000","last_active":"2024-09-11T09:29:23.195809+0000","last_peered":"2024-09-11T09:29:23.195809+0000","last_clean":"2024-09-11T09:29:23.195809+0000","last_became_active":"2024-09-11T09:28:46.699767+0000","last_became_peered":"2024-09-11T09:28:46.699767+0000","last_unstale":"2024-09-11T09:29:23.195809+0000","last_undegraded":"2024-09-11T09:29:23.195809+0000","last_fullsized":"2024-09-11T09:29:23.195809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:59:01.489464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085759499999999995,"stat_sum":{"num_bytes":9672208,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":3817,"num_read_kb":6579,"num_write":172,"num_write_kb":1018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":387868856,"num_objects":807,"num_object_clones":226,"num_object_copies":1614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":807,"num_whiteouts":241,"num_read":121504,"num_read_kb":475320,"num_write":10469,"num_write_kb":108375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":148526,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16404480,"data_stored":114913136,"data_compressed":4525009,"data_compressed_allocated":8548352,"data_compressed_original":108323840,"omap_allocated":0,"internal_metadata":0},"log_size":8856,"ondisk_log_size":8856,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17350,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607566,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49032,"kb_used_data":3960,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322808,"statfs":{"total":96636764160,"available":96586555392,"internally_reserved":0,"allocated":4055040,"data_stored":27504893,"data_compressed":891267,"data_compressed_allocated":1871872,"data_compressed_original":25884672,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607566,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46984,"kb_used_data":3316,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94324856,"statfs":{"total":96636764160,"available":96588652544,"internally_reserved":0,"allocated":3395584,"data_stored":17921345,"data_compressed":936849,"data_compressed_allocated":1482752,"data_compressed_original":16650240,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43356,"kb_used_data":3852,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328484,"statfs":{"total":96636764160,"available":96592367616,"internally_reserved":0,"allocated":3944448,"data_stored":25676638,"data_compressed":868885,"data_compressed_allocated":1908736,"data_compressed_original":24261120,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607567,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607567,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52716,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319124,"statfs":{"total":96636764160,"available":96582782976,"internally_reserved":0,"allocated":3559424,"data_stored":20037101,"data_compressed":946763,"data_compressed_allocated":1581056,"data_compressed_original":18553856,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640269,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42892,"kb_used_data":3384,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328948,"statfs":{"total":96636764160,"available":96592842752,"internally_reserved":0,"allocated":3465216,"data_stored":25265497,"data_compressed":898174,"data_compressed_allocated":1748992,"data_compressed_original":24075776,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3137536,"data_stored":24727684,"data_compressed":889783,"data_compressed_allocated":1728512,"data_compressed_original":23539200,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3108864,"data_stored":19914369,"data_compressed":946830,"data_compressed_allocated":1585152,"data_compressed_original":18562048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3481600,"data_stored":24974652,"data_compressed":860079,"data_compressed_allocated":1867776,"data_compressed_original":23663104,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3035136,"data_stored":17870655,"data_compressed":936983,"data_compressed_allocated":1490944,"data_compressed_original":16666624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3641344,"data_stored":27425776,"data_compressed":891334,"data_compressed_allocated":1875968,"data_compressed_original":25892864,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:29:25.651 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T09:29:25.651 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:25.651 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 3, 4, 5] 2024-09-11T09:29:25.651 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-11T09:29:25.658 INFO:teuthology.orchestra.run.smithi107.stdout:3036 write 0xebe8a00 thru 0xebef9df (0x6fe0 bytes) 2024-09-11T09:29:25.658 INFO:teuthology.orchestra.run.smithi107.stdout:3038 write 0x4548200 thru 0x455204e (0x9e4f bytes) 2024-09-11T09:29:25.659 INFO:teuthology.orchestra.run.smithi107.stdout:3039 write 0xb01c000 thru 0xb02716b (0xb16c bytes) 2024-09-11T09:29:25.659 INFO:teuthology.orchestra.run.smithi107.stdout:3042 punch from 0x2a9e600 to 0x2aa0800, (0x2200 bytes) 2024-09-11T09:29:25.660 INFO:teuthology.orchestra.run.smithi107.stdout:3043 read 0x39a8e00 thru 0x39b63cb (0xd5cc bytes) 2024-09-11T09:29:25.662 INFO:teuthology.orchestra.run.smithi107.stdout:3044 trunc from 0xebefa00 to 0xdcab800 2024-09-11T09:29:25.681 INFO:teuthology.orchestra.run.smithi107.stdout:3047 write 0xc75b200 thru 0xc76ac93 (0xfa94 bytes) 2024-09-11T09:29:25.682 INFO:teuthology.orchestra.run.smithi107.stdout:3049 write 0x86da800 thru 0x86e1dc4 (0x75c5 bytes) 2024-09-11T09:29:25.682 INFO:teuthology.orchestra.run.smithi107.stdout:3050 write 0x92bd400 thru 0x92cc6a2 (0xf2a3 bytes) 2024-09-11T09:29:25.682 INFO:teuthology.orchestra.run.smithi107.stdout:3052 read 0x8f52a00 thru 0x8f5da7d (0xb07e bytes) 2024-09-11T09:29:25.682 INFO:teuthology.orchestra.run.smithi107.stdout:3053 punch from 0xb480400 to 0xb48c600, (0xc200 bytes) 2024-09-11T09:29:25.683 INFO:teuthology.orchestra.run.smithi107.stdout:3055 write 0xb72c00 thru 0xb7be41 (0x9242 bytes) 2024-09-11T09:29:25.684 INFO:teuthology.orchestra.run.smithi107.stdout:3059 write 0x8cbc600 thru 0x8cc2f9b (0x699c bytes) 2024-09-11T09:29:25.684 INFO:teuthology.orchestra.run.smithi107.stdout:3061 read 0xd5ef200 thru 0xd5ef45e (0x25f bytes) 2024-09-11T09:29:25.684 INFO:teuthology.orchestra.run.smithi107.stdout:3062 read 0x596400 thru 0x5a18f8 (0xb4f9 bytes) 2024-09-11T09:29:25.688 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:25.688 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:25.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:25.688 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:25.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.776821129, 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:25.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:25.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:25.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.776590557, 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.776493352999999, 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:25.699 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:25.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:25.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.704 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.776258588999999, 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:25.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:25.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.776062573000001, 2024-09-11T09:29:25.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:25.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.775794136, 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:25.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:25.718 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.775692385999999, 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.719 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:25.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:25.723 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.775298728999999, 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:25.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.775172484, 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.728 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:25.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.774963101999999, 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.733 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:25.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.774852152999999, 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.742 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.774596215000001, 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:25.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.774443686, 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.747 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:25.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.774251322, 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:25.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:25.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:25.756 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.773920904000001, 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:25.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:25.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:25.761 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.773799351999999, 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:25.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:25.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.773644574, 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:25.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.773590653999999, 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.771 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:25.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.775 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.773390084999999, 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:25.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.780 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.773034546, 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:25.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:25.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:25.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:25.785 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:25.787 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:25.787+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:3064 punch from 0x5636400 to 0x563e400, (0x8000 bytes) 2024-09-11T09:29:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:3066 read 0x8885800 thru 0x8892f15 (0xd716 bytes) 2024-09-11T09:29:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:3067 read 0x60a2e00 thru 0x60a7d98 (0x4f99 bytes) 2024-09-11T09:29:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:3069 read 0xad7dc00 thru 0xad848e4 (0x6ce5 bytes) 2024-09-11T09:29:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:3071 read 0x6d0ec00 thru 0x6d15814 (0x6c15 bytes) 2024-09-11T09:29:25.788 INFO:teuthology.orchestra.run.smithi107.stdout:3072 clone 17 order 22 su 65536 sc 9 2024-09-11T09:29:25.789 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:29:25.887 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:25.888+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:25.904 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone16 from 0xdcab800 (overlap 0x10aa000) to 0x355200 2024-09-11T09:29:25.947 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:25.947 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:25.947 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:25.957 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:25.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-11T09:29:25.963+0000 7fb9f1e53640 0 --2- 172.21.15.17:0/2971884815 >> v2:172.21.15.17:3300/0 conn(0x7fb9ec135120 0x7fb9ec135500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-11T09:29:25.988 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:25.989+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:26.072 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:26.072 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:26.072 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:26.072 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:26.072 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:26.081 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:26.122 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:26.123+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:26.171 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-11T09:29:26.171+0000 7fef02752640 -1 mon.a@0(leader).osd e52 definitely_dead 0 2024-09-11T09:29:26.190 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:26.190+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:26.206 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:26.229 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 49.770593499, 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:26.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:26.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 37.509073811, 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:26.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:26.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:26.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:26.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:26.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:26.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 37.507827843999998, 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:26.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:26.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.245 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:26.245 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:26.245 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:26.246 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:26.290 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:26.291+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:26.331 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:26.349 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:26.350 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:26.354 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:26.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:26.354 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:26.355 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:26.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:26.365 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:26.366 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:26.383 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:26.383 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:26.400 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:26.400 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:26.405 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:26.406+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:26.417 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:26.417 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:26.419 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-11T09:29:26.493 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:26.494+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:26.523 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:26.523 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:26.535 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:26.594 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:26.595+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:26.695 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:26.695+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:26.712 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:26.712 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:26.722 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:26.796 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:26.796+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:26.839 INFO:teuthology.orchestra.run.smithi017.stderr:marked out osd.3. 2024-09-11T09:29:26.854 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:26.854 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:26.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:26.904 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:26.904+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:26.997 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:26.998+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:27.005 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.005 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.014 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:27.098 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:27.099+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:27.117 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.117 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.126 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:27.199 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:27.200+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:27.269 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.269 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:27.321 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:27.322+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:27.379 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.379 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.387 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:27.401 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:27.402+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:27.502 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:27.503+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:27.530 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.530 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:27.604 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:27.604+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:27.653 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.653 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:27.704 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:27.704+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:27.805 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:27.806+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:27.810 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.810 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.818 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:27.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:29:27.906 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:27.906+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:27.926 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:27.926 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:27.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:28.007 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:28.007+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:28.083 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:28.084 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:28.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:28.108 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:28.108+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:28.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:28.210 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:28.209+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:28.310 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:28.310+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:28.316 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:28.316 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:28.325 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:28.411 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:28.412+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:28.429 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:28.474 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:28.474 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:28.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:28.512 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:28.512+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:28.553 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:28.553 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:28.570 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:28.570 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:28.587 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:28.587 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:28.592 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:28.592 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:28.592 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:28.592 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:28.592 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:28.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:28.605 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:28.605 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:28.613 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:28.613+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:28.622 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:28.623 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:28.640 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:28.640 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:29:28.732 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:28.733 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:28.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:28.733 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:28.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.733 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:28.733 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:28.734 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 52.272662519000001, 2024-09-11T09:29:28.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:28.734 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:28.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:28.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:28.735 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:28.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:28.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:28.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:28.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:28.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:28.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:28.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:28.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:28.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:28.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:28.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:28.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:28.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:28.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:28.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:28.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:28.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:28.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:28.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:28.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:28.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:28.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:28.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:28.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:28.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:28.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:28.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:28.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:28.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:28.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:28.743 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.744 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:28.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.744 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:28.744 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:28.745 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 40.011142831000001, 2024-09-11T09:29:28.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:28.745 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:28.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:28.745 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:28.746 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:28.746 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:28.746 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:28.746 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:28.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:28.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:28.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:28.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:28.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:28.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:28.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:28.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 40.009896863999998, 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:28.754 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:28.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:28.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:28.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:28.759 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:28.760 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:28.733+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:28.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:28.815 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:28.815+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:28.894 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:28.895 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:28.895 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:28.895 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:28.904 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:28.916 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:28.916+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:28.958 INFO:teuthology.orchestra.run.smithi107.stdout:3074 read 0x9c57400 thru 0x9c58bec (0x17ed bytes) 2024-09-11T09:29:28.959 INFO:teuthology.orchestra.run.smithi107.stdout:3076 write 0xc8cae00 thru 0xc8d8a2f (0xdc30 bytes) 2024-09-11T09:29:28.960 INFO:teuthology.orchestra.run.smithi107.stdout:3077 read 0x8f4ec00 thru 0x8f553ac (0x67ad bytes) 2024-09-11T09:29:28.961 INFO:teuthology.orchestra.run.smithi107.stdout:3078 read 0x4eaf000 thru 0x4eb7f45 (0x8f46 bytes) 2024-09-11T09:29:28.962 INFO:teuthology.orchestra.run.smithi107.stdout:3079 write 0x64aa00 thru 0x6511a7 (0x67a8 bytes) 2024-09-11T09:29:28.973 INFO:teuthology.orchestra.run.smithi107.stdout:3080 trunc from 0xdcab800 to 0x58a8400 2024-09-11T09:29:29.017 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:29.017+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:29.064 INFO:teuthology.orchestra.run.smithi107.stdout:3081 read 0x4131e00 thru 0x413a4f2 (0x86f3 bytes) 2024-09-11T09:29:29.071 INFO:teuthology.orchestra.run.smithi107.stdout:3082 trunc from 0x58a8400 to 0x597c400 2024-09-11T09:29:29.081 INFO:teuthology.orchestra.run.smithi107.stdout:3087 read 0x3d56e00 thru 0x3d5b31e (0x451f bytes) 2024-09-11T09:29:29.084 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:29.084 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:29.085 INFO:teuthology.orchestra.run.smithi107.stdout:3090 write 0x5875600 thru 0x588348d (0xde8e bytes) 2024-09-11T09:29:29.085 INFO:teuthology.orchestra.run.smithi107.stdout:3094 read 0x2bdd000 thru 0x2bdef62 (0x1f63 bytes) 2024-09-11T09:29:29.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:29.107 INFO:teuthology.orchestra.run.smithi107.stdout:3095 write 0x9e07c00 thru 0x9e15270 (0xd671 bytes) 2024-09-11T09:29:29.107 INFO:teuthology.orchestra.run.smithi107.stdout:3096 read 0x9655000 thru 0x9660598 (0xb599 bytes) 2024-09-11T09:29:29.107 INFO:teuthology.orchestra.run.smithi107.stdout:3097 write 0x5d7fa00 thru 0x5d8da9d (0xe09e bytes) 2024-09-11T09:29:29.108 INFO:teuthology.orchestra.run.smithi107.stdout:3100 write 0x2a3ac00 thru 0x2a3e971 (0x3d72 bytes) 2024-09-11T09:29:29.108 INFO:teuthology.orchestra.run.smithi107.stdout:3101 punch from 0x6c1600 to 0x6cf400, (0xde00 bytes) 2024-09-11T09:29:29.111 INFO:teuthology.orchestra.run.smithi107.stdout:3102 trunc from 0x9e15400 to 0x8b3fc00 2024-09-11T09:29:29.117 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:29.118+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:29.148 INFO:teuthology.orchestra.run.smithi107.stdout:3105 trunc from 0x8b3fc00 to 0x6186a00 2024-09-11T09:29:29.160 INFO:teuthology.orchestra.run.smithi107.stdout:3106 read 0x1222000 thru 0x122d9bf (0xb9c0 bytes) 2024-09-11T09:29:29.162 INFO:teuthology.orchestra.run.smithi107.stdout:3107 read 0x2e4fa00 thru 0x2e53d46 (0x4347 bytes) 2024-09-11T09:29:29.162 INFO:teuthology.orchestra.run.smithi107.stdout:3108 read 0x360e200 thru 0x3612e82 (0x4c83 bytes) 2024-09-11T09:29:29.169 INFO:teuthology.orchestra.run.smithi107.stdout:3109 trunc from 0x6186a00 to 0xdf1200 2024-09-11T09:29:29.218 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:29.219+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:29.235 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:29.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.797836435999997, 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:29.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:29.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5081 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.242 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273561242, 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043672824999999998, 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5081 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:29.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961281+0000", 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961285+0000", 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3560000000000001e-06 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961287+0000", 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5409999999999999e-06 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004781+0000", 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043493472999999998 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004856+0000", 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5741000000000005e-05 2024-09-11T09:29:29.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004947+0000", 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1222000000000002e-05 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5083 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273545668000001, 2024-09-11T09:29:29.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043785219, 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5083 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:29.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961291+0000", 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961292+0000", 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961295+0000", 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0149999999999999e-06 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961297+0000", 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004977+0000", 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043679791000000003 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005021+0000", 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4295000000000002e-05 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005075+0000", 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999998e-05 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5088 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 905216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273536120999999, 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043963918999999997, 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5088 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961301+0000", 2024-09-11T09:29:29.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.13e-06 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961305+0000", 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4529999999999998e-06 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961309+0000", 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005096+0000", 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043787848999999997 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005175+0000", 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8487999999999998e-05 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005264+0000", 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8786e-05 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5097 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1605632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273499273999999, 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044049360000000003, 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5097 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961339+0000", 2024-09-11T09:29:29.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1440000000000001e-06 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961345+0000", 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7470000000000002e-06 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961350+0000", 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209999999999996e-06 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005291+0000", 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043940405000000002 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005332+0000", 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1542999999999998e-05 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005386+0000", 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999998e-05 2024-09-11T09:29:29.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5100 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273479423000001, 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044229475999999997, 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5100 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961357+0000", 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961359+0000", 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.052e-06 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961365+0000", 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961369+0000", 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6689999999999997e-06 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005422+0000", 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044052701 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005499+0000", 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6664999999999999e-05 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005586+0000", 2024-09-11T09:29:29.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7230000000000003e-05 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5105 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273460841999999, 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044370891000000003, 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5105 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961377+0000", 2024-09-11T09:29:29.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9120000000000001e-06 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961383+0000", 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961388+0000", 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005629+0000", 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044240743999999999 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005687+0000", 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7951000000000001e-05 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005746+0000", 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9444999999999998e-05 2024-09-11T09:29:29.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5108 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1607680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273442485, 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581704, 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5108 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961394+0000", 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961396+0000", 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961401+0000", 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8119999999999996e-06 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961406+0000", 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4920000000000004e-06 2024-09-11T09:29:29.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005789+0000", 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383498 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005882+0000", 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2541999999999996e-05 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005975+0000", 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3110999999999997e-05 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5111 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273424441, 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044694581999999997, 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5111 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961412+0000", 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961414+0000", 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.996e-06 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961420+0000", 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8000000000000004e-06 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961425+0000", 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5149999999999997e-06 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006007+0000", 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581786999999998 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006051+0000", 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3998999999999997e-05 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006106+0000", 2024-09-11T09:29:29.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5145999999999997e-05 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5116 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1632256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273405037, 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044886862999999999, 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5116 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961433+0000", 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:29:29.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961439+0000", 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961444+0000", 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809999999999998e-06 2024-09-11T09:29:29.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006144+0000", 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044699782 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006221+0000", 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7270999999999999e-05 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006318+0000", 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6914000000000001e-05 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5121 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1611776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273386616, 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045003543, 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5121 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961450+0000", 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961452+0000", 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0030000000000001e-06 2024-09-11T09:29:29.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961459+0000", 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961464+0000", 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006348+0000", 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044884563000000002 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006396+0000", 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7817e-05 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006453+0000", 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6762000000000003e-05 2024-09-11T09:29:29.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5124 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273366147000001, 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045118734000000001, 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5124 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961472+0000", 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929999999999999e-06 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961478+0000", 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.592e-06 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961482+0000", 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006470+0000", 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044987680000000002 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006529+0000", 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9116000000000001e-05 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006589+0000", 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9463000000000003e-05 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5129 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1624064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273347778000002, 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045244390000000002, 2024-09-11T09:29:29.297 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5129 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000002e-07 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961490+0000", 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:29.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961496+0000", 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961500+0000", 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3819999999999997e-06 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006606+0000", 2024-09-11T09:29:29.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045106159 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006669+0000", 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2188999999999997e-05 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006733+0000", 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3848000000000003e-05 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5134 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273312807, 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045344917999999998, 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5134 2024-09-11T09:29:29.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961524+0000", 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961531+0000", 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961541+0000", 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0059e-05 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961546+0000", 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006750+0000", 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045204167000000003 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006808+0000", 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8229000000000001e-05 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006868+0000", 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9953e-05 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5135 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1622016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.306 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273283106000001, 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045554071000000002, 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5135 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7100000000000002e-07 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961555+0000", 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1679999999999998e-06 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961561+0000", 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961566+0000", 2024-09-11T09:29:29.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4170000000000002e-06 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006908+0000", 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045342423999999999 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007004+0000", 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5307000000000001e-05 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007107+0000", 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103253 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5138 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1634304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.273263336999999, 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045679006000000001, 2024-09-11T09:29:29.311 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5138 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:29.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961805+0000", 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023282700000000001 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961816+0000", 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961828+0000", 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2318e-05 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961835+0000", 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007139+0000", 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045303738000000003 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007188+0000", 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9014999999999997e-05 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007252+0000", 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3921e-05 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5143 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1613824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.272989953, 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045634479999999998, 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5143 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961994+0000", 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147949 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961997+0000", 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5580000000000001e-06 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962007+0000", 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6579999999999997e-06 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962014+0000", 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007290+0000", 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045276416 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007380+0000", 2024-09-11T09:29:29.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9686999999999997e-05 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007480+0000", 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100762 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.321 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5148 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1626112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.272812063, 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045674845999999998, 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5148 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962412+0000", 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038829000000000001 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962417+0000", 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962427+0000", 2024-09-11T09:29:29.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962434+0000", 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7379999999999997e-06 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007514+0000", 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045080264000000002 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007600+0000", 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5970000000000005e-05 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007699+0000", 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8706e-05 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5153 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1615872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:29.326 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.272390651999999, 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045448919999999997, 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5153 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962446+0000", 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962448+0000", 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962455+0000", 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2024-09-11T09:29:29.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962458+0000", 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007712+0000", 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045253466999999999 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007784+0000", 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1939000000000004e-05 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007894+0000", 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5156 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:29.331 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.272371593999999, 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045635835999999999, 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5156 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962886+0000", 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042155799999999999 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962890+0000", 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962899+0000", 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0440000000000001e-06 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962906+0000", 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1169999999999998e-06 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007917+0000", 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045011412000000001 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007994+0000", 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6650999999999995e-05 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.008100+0000", 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106164 2024-09-11T09:29:29.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.336 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:29.337 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:29.337+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:29.338 INFO:teuthology.orchestra.run.smithi107.stdout:3110 write 0x32a3200 thru 0x32b1860 (0xe661 bytes) 2024-09-11T09:29:29.338 INFO:teuthology.orchestra.run.smithi107.stdout:3113 read 0x279b800 thru 0x279ba55 (0x256 bytes) 2024-09-11T09:29:29.338 INFO:teuthology.orchestra.run.smithi107.stdout:3115 read 0x2906400 thru 0x29133d7 (0xcfd8 bytes) 2024-09-11T09:29:29.338 INFO:teuthology.orchestra.run.smithi107.stdout:3117 write 0xd8c600 thru 0xd91c9c (0x569d bytes) 2024-09-11T09:29:29.338 INFO:teuthology.orchestra.run.smithi107.stdout:3118 clone 18 order 18 su 2048 sc 4 2024-09-11T09:29:29.339 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:29.438 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:29.438+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:29.443 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:29.443 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:29.443 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:29.452 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:29.539 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:29.539+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:29.594 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:29.595 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:29.595 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:29.595 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:29.595 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:29.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:29.644 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:29.644+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:29.741 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:29.742+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.384139208000001, 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.384101823000002, 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.384082504999999, 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.384071258999999, 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.384029353999999, 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.384011950000001, 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:29.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:29.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:29.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383994054999999, 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:29.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383973874999999, 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:29.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383952891, 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:29.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:29.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:29.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:29.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383925794, 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:29.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:29.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383906506999999, 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:29.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:29.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383702147000001, 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:29.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:29.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383168391000002, 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:29.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:29.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:29.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383119497999999, 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383071409999999, 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:29.816 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:29.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:29.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.383033372, 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:29.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.382809529999999, 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:29.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:29.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.830 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.382632428000001, 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:29.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.382486873000001, 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:29.835 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:29.839 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.382365487000001, 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:29.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:29.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:29.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.843 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.844 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:29.845 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:29.845 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:29.845+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:29.949 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:29.950+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3.e9e2906f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.29489189700000001, 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.294913441, 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for peered", 2024-09-11T09:29:29.987 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:29.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:29:29.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-11T09:29:29.991 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:29.995 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:30.050 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:30.051+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:30.097 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:30.097 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:30.097 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:30.097 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:30.098 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:30.106 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:30.151 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:30.152+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:30.210 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:30.210 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4835 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.227069216, 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033722624999999999, 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4835 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983043+0000", 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7800000000000002e-07 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983045+0000", 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.657e-06 2024-09-11T09:29:30.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983049+0000", 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3420000000000001e-06 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983052+0000", 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.74e-06 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016669+0000", 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033617247000000003 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016711+0000", 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1149999999999997e-05 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016766+0000", 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5111000000000001e-05 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.227013367000001, 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:30.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.226991592000001, 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:30.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:30.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.226945548, 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:30.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.226933878000001, 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:30.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.226060469, 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:30.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.240 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225696994, 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:30.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:30.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225645609000001, 2024-09-11T09:29:30.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:30.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225609907999999, 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:30.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225587870999998, 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:30.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:30.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:30.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225552285999999, 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225491943000002, 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225443258999999, 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:30.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:30.273 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225386794999999, 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:30.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225367240000001, 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:30.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:30.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225319476999999, 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:30.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225205739, 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:30.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:30.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225186995000001, 2024-09-11T09:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:30.296 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225148006000001, 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.297 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:30.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.301 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.225124899000001, 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:30.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:30.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:30.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:30.306 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:30.308 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:30.308+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:30.308 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:30.408 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:30.409+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:30.428 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:30.428 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:30.428 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:30.428 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:30.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:30.509 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:30.509+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:30.546 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:30.546 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:30.546 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:30.546 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:30.546 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:30.554 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:30.638 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:30.639+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:30.651 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:30.707 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:29:30.711 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:30.711 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:30.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:30.711 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:30.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 29.039463047000002, 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:30.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:30.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:30.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1715 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834239383, 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022344029000000001, 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.718 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1715 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.96e-07 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876407+0000", 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.871e-06 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876414+0000", 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4679999999999997e-06 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876419+0000", 2024-09-11T09:29:30.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1939999999999997e-06 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898563+0000", 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022144646 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898643+0000", 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9900000000000004e-05 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898749+0000", 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105554 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1716 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834219122, 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022552065999999999, 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1716 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876426+0000", 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.77e-07 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876432+0000", 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0079999999999999e-06 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876438+0000", 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4999999999999996e-06 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876443+0000", 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.259e-06 2024-09-11T09:29:30.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898781+0000", 2024-09-11T09:29:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022337308 2024-09-11T09:29:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898875+0000", 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4388000000000005e-05 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898977+0000", 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010222599999999999 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1717 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834194217, 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022753961, 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1717 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.729 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876451+0000", 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876452+0000", 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.905e-06 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876459+0000", 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4980000000000003e-06 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876464+0000", 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1909999999999999e-06 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899008+0000", 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022544281999999999 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899093+0000", 2024-09-11T09:29:30.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4505000000000005e-05 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899204+0000", 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111049 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1718 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:30.733 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834173552999999, 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02296196, 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1718 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:30.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876473+0000", 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9259999999999999e-06 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876480+0000", 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876485+0000", 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2159999999999997e-06 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899235+0000", 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022750631 2024-09-11T09:29:30.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899321+0000", 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5958999999999995e-05 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899433+0000", 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011126899999999999 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1719 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 262144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834133135999998, 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023149355, 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1719 2024-09-11T09:29:30.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:30.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876512+0000", 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.44e-07 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876514+0000", 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:30.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876522+0000", 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5000000000000002e-06 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876527+0000", 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.587e-06 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899467+0000", 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022939975000000001 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899552+0000", 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5131999999999997e-05 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899660+0000", 2024-09-11T09:29:30.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108137 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1720 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834109546000001, 2024-09-11T09:29:30.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023387654000000001, 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1720 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.63e-07 2024-09-11T09:29:30.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876537+0000", 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1040000000000001e-06 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876544+0000", 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7800000000000003e-06 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876549+0000", 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2549999999999997e-06 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899706+0000", 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023156708000000002 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899801+0000", 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4826000000000003e-05 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899922+0000", 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121518 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1760 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 12288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834088653999999, 2024-09-11T09:29:30.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023618561999999999, 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1760 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876558+0000", 2024-09-11T09:29:30.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1670000000000002e-06 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876566+0000", 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3810000000000001e-06 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876571+0000", 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3029999999999999e-06 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899963+0000", 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023392445000000001 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900061+0000", 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7495000000000001e-05 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900174+0000", 2024-09-11T09:29:30.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113391 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1761 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834066854, 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023824222999999999, 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.754 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1761 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876578+0000", 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876580+0000", 2024-09-11T09:29:30.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9539999999999998e-06 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876586+0000", 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876591+0000", 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900220+0000", 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023628546 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900317+0000", 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7168000000000002e-05 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900402+0000", 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4701000000000001e-05 2024-09-11T09:29:30.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1762 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 57344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834046599000001, 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024033523000000001, 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.759 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1762 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876600+0000", 2024-09-11T09:29:30.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.052e-06 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876607+0000", 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876612+0000", 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9259999999999999e-06 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900448+0000", 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023836570000000001 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900534+0000", 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5314000000000003e-05 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900631+0000", 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7676000000000005e-05 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1763 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 79872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834026173000002, 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024166084000000001, 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.764 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1763 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876619+0000", 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6400000000000003e-07 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876621+0000", 2024-09-11T09:29:30.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0949999999999998e-06 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876629+0000", 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8380000000000008e-06 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876635+0000", 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3199999999999999e-06 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900675+0000", 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024040611 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900724+0000", 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8097999999999999e-05 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900784+0000", 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0658000000000001e-05 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1764 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.834002936000001, 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024356692999999999, 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.769 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1764 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876642+0000", 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876644+0000", 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.041e-06 2024-09-11T09:29:30.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876651+0000", 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6309999999999997e-06 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876656+0000", 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999999e-06 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900814+0000", 2024-09-11T09:29:30.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024158129 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900906+0000", 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2618e-05 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900998+0000", 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1624000000000002e-05 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1765 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833982206000002, 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024574166000000001, 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1765 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.774 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.39e-07 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876664+0000", 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9520000000000001e-06 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876671+0000", 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3670000000000002e-06 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876676+0000", 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901029+0000", 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024353631000000001 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901109+0000", 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9951999999999996e-05 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901236+0000", 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012697000000000001 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833962416999999, 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.778 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:30.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833942185000002, 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:30.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833916059, 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833893774, 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.793 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:30.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833870334, 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:30.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:30.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833850486999999, 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:30.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 24.833818109999999, 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:30.808 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:30.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:30.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:30.813 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:30.814 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:30.711+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:30.815 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:30.815 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:30.816 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone17 from 0x32b1a00 (overlap 0xdf1200) to 0x2c9c00 2024-09-11T09:29:30.817 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:30.817+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:30.823 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:30.823 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:30.840 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:30.841 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:30.872 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:30.873 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:30.905 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:30.905 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:30.917 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:30.918+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:30.931 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:30.931 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:30.942 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:30.942 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:30.952 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:31.035 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:31.036+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:31.104 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:31.104 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:31.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:31.114 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-11T09:29:31.154 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:31.155+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:31.190 INFO:teuthology.orchestra.run.smithi107.stdout:3120 read 0x1f47200 thru 0x1f4f17b (0x7f7c bytes) 2024-09-11T09:29:31.193 INFO:teuthology.orchestra.run.smithi107.stdout:3121 read 0x2ea0400 thru 0x2eaf463 (0xf064 bytes) 2024-09-11T09:29:31.194 INFO:teuthology.orchestra.run.smithi107.stdout:3125 read 0x2b5e400 thru 0x2b5f277 (0xe78 bytes) 2024-09-11T09:29:31.195 INFO:teuthology.orchestra.run.smithi107.stdout:3126 read 0x1e8a400 thru 0x1e98282 (0xde83 bytes) 2024-09-11T09:29:31.198 INFO:teuthology.orchestra.run.smithi107.stdout:3129 read 0x2e6e400 thru 0x2e7623e (0x7e3f bytes) 2024-09-11T09:29:31.201 INFO:teuthology.orchestra.run.smithi107.stdout:3131 trunc from 0x32b1a00 to 0x545d200 2024-09-11T09:29:31.212 INFO:teuthology.orchestra.run.smithi107.stdout:3132 read 0x20e0a00 thru 0x20edac9 (0xd0ca bytes) 2024-09-11T09:29:31.219 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:31.220+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:31.234 INFO:teuthology.orchestra.run.smithi107.stdout:3133 write 0x684e400 thru 0x6857d98 (0x9999 bytes) 2024-09-11T09:29:31.261 INFO:teuthology.orchestra.run.smithi107.stdout:3134 write 0xc1f9e00 thru 0xc1fb340 (0x1541 bytes) 2024-09-11T09:29:31.274 INFO:teuthology.orchestra.run.smithi107.stdout:3135 trunc from 0xc1fb400 to 0x2097e00 2024-09-11T09:29:31.286 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:31.286 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:31.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:31.320 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:31.321+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:31.334 INFO:teuthology.orchestra.run.smithi107.stdout:3138 trunc from 0x2097e00 to 0x7689800 2024-09-11T09:29:31.344 INFO:teuthology.orchestra.run.smithi107.stdout:3139 read 0x207be00 thru 0x207ddf4 (0x1ff5 bytes) 2024-09-11T09:29:31.346 INFO:teuthology.orchestra.run.smithi107.stdout:3141 trunc from 0x7689800 to 0x6308a00 2024-09-11T09:29:31.363 INFO:teuthology.orchestra.run.smithi107.stdout:3142 read 0x1a7a800 thru 0x1a7b4b9 (0xcba bytes) 2024-09-11T09:29:31.376 INFO:teuthology.orchestra.run.smithi107.stdout:3143 trunc from 0x6308a00 to 0x49df600 2024-09-11T09:29:31.384 INFO:teuthology.orchestra.run.smithi107.stdout:3144 write 0x13a3800 thru 0x13a5a01 (0x2202 bytes) 2024-09-11T09:29:31.390 INFO:teuthology.orchestra.run.smithi107.stdout:3145 read 0xa0a400 thru 0xa0f924 (0x5525 bytes) 2024-09-11T09:29:31.392 INFO:teuthology.orchestra.run.smithi107.stdout:3146 write 0x1234000 thru 0x123468e (0x68f bytes) 2024-09-11T09:29:31.392 INFO:teuthology.orchestra.run.smithi107.stdout:3147 read 0x491200 thru 0x498116 (0x6f17 bytes) 2024-09-11T09:29:31.400 INFO:teuthology.orchestra.run.smithi107.stdout:3151 trunc from 0x49df600 to 0x59bae00 2024-09-11T09:29:31.422 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:31.422+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:31.447 INFO:teuthology.orchestra.run.smithi107.stdout:3152 read 0x44bc600 thru 0x44c8e37 (0xc838 bytes) 2024-09-11T09:29:31.449 INFO:teuthology.orchestra.run.smithi107.stdout:3153 trunc from 0x59bae00 to 0x4686a00 2024-09-11T09:29:31.464 INFO:teuthology.orchestra.run.smithi107.stdout:3154 punch from 0x2640400 to 0x2649800, (0x9400 bytes) 2024-09-11T09:29:31.465 INFO:teuthology.orchestra.run.smithi107.stdout:3156 read 0x1887800 thru 0x1891c29 (0xa42a bytes) 2024-09-11T09:29:31.469 INFO:teuthology.orchestra.run.smithi107.stdout:3158 clone 19 order 22 su 32768 sc 10 2024-09-11T09:29:31.474 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:31.474 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:31.486 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:31.524 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:31.524+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:31.624 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:31.625+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:31.632 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:31.632 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:31.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:29:31.725 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:31.725+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:31.754 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:31.754 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:31.763 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:31.790 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone18 2024-09-11T09:29:31.825 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:31.826+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:31.855 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:31.855 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:31.855 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-11T09:29:31.855 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-11T09:29:31.878 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:31.878 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:31.887 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:31.928 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:31.928+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:31.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-11T09:29:32.035 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:32.036+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:32.039 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.039 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:32.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:32.127 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:32.128+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:32.192 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.192 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:32.203 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:32.229 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-11T09:29:32.243 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:32.244+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:32.329 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:32.329+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:32.377 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.377 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:32.389 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:32.430 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:32.430+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:32.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-11T09:29:32.530 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:32.531+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:32.555 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.555 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:32.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:32.650 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:32.651+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:32.730 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.731 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:32.731 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:32.732+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:32.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:32.832 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:32.833+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:32.884 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.884 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:32.893 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:32.936 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:32.937+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:32.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:32.998 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:32.999 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:33.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:33.048 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:33.048+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:33.057 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-11T09:29:33.076 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:33.076 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:33.093 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:33.093 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:33.109 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:33.109 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:33.126 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:33.126 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:33.129 INFO:teuthology.orchestra.run.smithi107.stdout:3159 read 0x4d1a00 thru 0x4d4bd9 (0x31da bytes) 2024-09-11T09:29:33.129 INFO:teuthology.orchestra.run.smithi107.stdout:3162 read 0x1525c00 thru 0x1531630 (0xba31 bytes) 2024-09-11T09:29:33.130 INFO:teuthology.orchestra.run.smithi107.stdout:3165 read 0x3320400 thru 0x3325508 (0x5109 bytes) 2024-09-11T09:29:33.130 INFO:teuthology.orchestra.run.smithi107.stdout:3166 read 0x1b3fc00 thru 0x1b462fd (0x66fe bytes) 2024-09-11T09:29:33.130 INFO:teuthology.orchestra.run.smithi107.stdout:3167 write 0x262da00 thru 0x263cdf3 (0xf3f4 bytes) 2024-09-11T09:29:33.131 INFO:teuthology.orchestra.run.smithi107.stdout:3174 read 0x44f7a00 thru 0x44fb560 (0x3b61 bytes) 2024-09-11T09:29:33.135 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:33.136+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:33.143 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:33.143 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:33.153 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:33.153 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:33.166 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:33.169 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:33.169 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:33.194 INFO:teuthology.orchestra.run.smithi107.stdout:3176 write 0xea2b400 thru 0xea2c371 (0xf72 bytes) 2024-09-11T09:29:33.194 INFO:teuthology.orchestra.run.smithi107.stdout:3179 write 0x23a800 thru 0x2473b1 (0xcbb2 bytes) 2024-09-11T09:29:33.195 INFO:teuthology.orchestra.run.smithi107.stdout:3180 trunc from 0xea2c400 to 0xec4e400 2024-09-11T09:29:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:3181 read 0x9512200 thru 0x951e2e5 (0xc0e6 bytes) 2024-09-11T09:29:33.213 INFO:teuthology.orchestra.run.smithi107.stdout:3182 read 0xbe84200 thru 0xbe8c74f (0x8550 bytes) 2024-09-11T09:29:33.214 INFO:teuthology.orchestra.run.smithi107.stdout:3183 read 0x193800 thru 0x19a2f0 (0x6af1 bytes) 2024-09-11T09:29:33.214 INFO:teuthology.orchestra.run.smithi107.stdout:3184 punch from 0x3f9d000 to 0x3fa1c00, (0x4c00 bytes) 2024-09-11T09:29:33.216 INFO:teuthology.orchestra.run.smithi107.stdout:3188 write 0x6b2dc00 thru 0x6b36a1e (0x8e1f bytes) 2024-09-11T09:29:33.216 INFO:teuthology.orchestra.run.smithi107.stdout:3190 read 0x3f6600 thru 0x404414 (0xde15 bytes) 2024-09-11T09:29:33.220 INFO:teuthology.orchestra.run.smithi107.stdout:3191 trunc from 0xec4e400 to 0xc240600 2024-09-11T09:29:33.236 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:33.237+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:33.241 INFO:teuthology.orchestra.run.smithi107.stdout:3192 read 0x5b46c00 thru 0x5b55b63 (0xef64 bytes) 2024-09-11T09:29:33.242 INFO:teuthology.orchestra.run.smithi107.stdout:3195 punch from 0xa4c7e00 to 0xa4ca800, (0x2a00 bytes) 2024-09-11T09:29:33.243 INFO:teuthology.orchestra.run.smithi107.stdout:3196 write 0x6250600 thru 0x6259a63 (0x9464 bytes) 2024-09-11T09:29:33.244 INFO:teuthology.orchestra.run.smithi107.stdout:3197 read 0xbaad200 thru 0xbab49c7 (0x77c8 bytes) 2024-09-11T09:29:33.253 INFO:teuthology.orchestra.run.smithi107.stdout:3198 trunc from 0xc240600 to 0x3f74400 2024-09-11T09:29:33.281 INFO:teuthology.orchestra.run.smithi107.stdout:3200 punch from 0x26ff000 to 0x2705600, (0x6600 bytes) 2024-09-11T09:29:33.282 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:33.282 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:33.283 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:33.283 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:33.287 INFO:teuthology.orchestra.run.smithi107.stdout:3201 read 0x393be00 thru 0x393d98c (0x1b8d bytes) 2024-09-11T09:29:33.290 INFO:teuthology.orchestra.run.smithi107.stdout:3203 punch from 0x1a85a00 to 0x1a91000, (0xb600 bytes) 2024-09-11T09:29:33.294 INFO:teuthology.orchestra.run.smithi107.stdout:3208 read 0x625400 thru 0x629651 (0x4252 bytes) 2024-09-11T09:29:33.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:33.301 INFO:teuthology.orchestra.run.smithi107.stdout:3210 read 0x28cc600 thru 0x28d17ed (0x51ee bytes) 2024-09-11T09:29:33.307 INFO:teuthology.orchestra.run.smithi107.stdout:3211 read 0xce1000 thru 0xce7366 (0x6367 bytes) 2024-09-11T09:29:33.320 INFO:teuthology.orchestra.run.smithi107.stdout:3212 trunc from 0x3f74400 to 0xb111000 2024-09-11T09:29:33.334 INFO:teuthology.orchestra.run.smithi107.stdout:3214 read 0x736d800 thru 0x7374d6f (0x7570 bytes) 2024-09-11T09:29:33.334 INFO:teuthology.orchestra.run.smithi107.stdout:3215 write 0x4b9fe00 thru 0x4ba64c3 (0x66c4 bytes) 2024-09-11T09:29:33.335 INFO:teuthology.orchestra.run.smithi107.stdout:3216 read 0x78ee200 thru 0x78fa87f (0xc680 bytes) 2024-09-11T09:29:33.335 INFO:teuthology.orchestra.run.smithi107.stdout:3218 write 0x445d800 thru 0x446c258 (0xea59 bytes) 2024-09-11T09:29:33.335 INFO:teuthology.orchestra.run.smithi107.stdout:3220 read 0xa083a00 thru 0xa093177 (0xf778 bytes) 2024-09-11T09:29:33.336 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:33.337+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:33.349 INFO:teuthology.orchestra.run.smithi107.stdout:3221 write 0xb451600 thru 0xb452a9d (0x149e bytes) 2024-09-11T09:29:33.367 INFO:teuthology.orchestra.run.smithi107.stdout:3223 write 0xd0a4200 thru 0xd0a81cb (0x3fcc bytes) 2024-09-11T09:29:33.367 INFO:teuthology.orchestra.run.smithi107.stdout:3224 write 0x35afe00 thru 0x35bae47 (0xb048 bytes) 2024-09-11T09:29:33.368 INFO:teuthology.orchestra.run.smithi107.stdout:3226 write 0x6ff2400 thru 0x6ff820a (0x5e0b bytes) 2024-09-11T09:29:33.368 INFO:teuthology.orchestra.run.smithi107.stdout:3227 write 0x6440a00 thru 0x644a4d8 (0x9ad9 bytes) 2024-09-11T09:29:33.371 INFO:teuthology.orchestra.run.smithi107.stdout:3231 trunc from 0xd0a8200 to 0xa811800 2024-09-11T09:29:33.403 INFO:teuthology.orchestra.run.smithi107.stdout:3232 trunc from 0xa811800 to 0x537e600 2024-09-11T09:29:33.416 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:33.416 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:33.416 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:33.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:33.417 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:33.418 INFO:teuthology.orchestra.run.smithi107.stdout:3233 read 0x11b0400 thru 0x11b0eda (0xadb bytes) 2024-09-11T09:29:33.418 INFO:teuthology.orchestra.run.smithi107.stdout:3234 read 0x821a00 thru 0x8312cb (0xf8cc bytes) 2024-09-11T09:29:33.420 INFO:teuthology.orchestra.run.smithi107.stdout:3235 read 0x4ff1c00 thru 0x5000b38 (0xef39 bytes) 2024-09-11T09:29:33.421 INFO:teuthology.orchestra.run.smithi107.stdout:3236 write 0x265aa00 thru 0x265c19f (0x17a0 bytes) 2024-09-11T09:29:33.421 INFO:teuthology.orchestra.run.smithi107.stdout:3237 write 0x2c7bc00 thru 0x2c86af1 (0xaef2 bytes) 2024-09-11T09:29:33.421 INFO:teuthology.orchestra.run.smithi107.stdout:3238 read 0x25bae00 thru 0x25c18a9 (0x6aaa bytes) 2024-09-11T09:29:33.423 INFO:teuthology.orchestra.run.smithi107.stdout:3239 read 0x1dac800 thru 0x1db8417 (0xbc18 bytes) 2024-09-11T09:29:33.423 INFO:teuthology.orchestra.run.smithi107.stdout:3240 read 0x419b400 thru 0x41a4e72 (0x9a73 bytes) 2024-09-11T09:29:33.424 INFO:teuthology.orchestra.run.smithi107.stdout:3241 read 0x26b2400 thru 0x26bd211 (0xae12 bytes) 2024-09-11T09:29:33.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:33.426 INFO:teuthology.orchestra.run.smithi107.stdout:3247 punch from 0x3e9f400 to 0x3ea5a00, (0x6600 bytes) 2024-09-11T09:29:33.426 INFO:teuthology.orchestra.run.smithi107.stdout:3249 read 0x1466400 thru 0x1471b8c (0xb78d bytes) 2024-09-11T09:29:33.426 INFO:teuthology.orchestra.run.smithi107.stdout:3252 read 0x5271000 thru 0x527644b (0x544c bytes) 2024-09-11T09:29:33.426 INFO:teuthology.orchestra.run.smithi107.stdout:3255 read 0x4b13200 thru 0x4b21074 (0xde75 bytes) 2024-09-11T09:29:33.427 INFO:teuthology.orchestra.run.smithi107.stdout:3258 read 0x1212c00 thru 0x121f518 (0xc919 bytes) 2024-09-11T09:29:33.427 INFO:teuthology.orchestra.run.smithi107.stdout:3259 punch from 0x25d1a00 to 0x25d4c00, (0x3200 bytes) 2024-09-11T09:29:33.438 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:33.438+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:33.442 INFO:teuthology.orchestra.run.smithi107.stdout:3260 write 0xaf04800 thru 0xaf082c6 (0x3ac7 bytes) 2024-09-11T09:29:33.443 INFO:teuthology.orchestra.run.smithi107.stdout:3265 read 0x943c600 thru 0x94439ac (0x73ad bytes) 2024-09-11T09:29:33.443 INFO:teuthology.orchestra.run.smithi107.stdout:3266 read 0x600b000 thru 0x600f24c (0x424d bytes) 2024-09-11T09:29:33.447 INFO:teuthology.orchestra.run.smithi107.stdout:3267 punch from 0x80aea00 to 0x80b8a00, (0xa000 bytes) 2024-09-11T09:29:33.447 INFO:teuthology.orchestra.run.smithi107.stdout:3268 write 0x609d400 thru 0x60a03f7 (0x2ff8 bytes) 2024-09-11T09:29:33.448 INFO:teuthology.orchestra.run.smithi107.stdout:3269 write 0x7e72800 thru 0x7e72f8a (0x78b bytes) 2024-09-11T09:29:33.448 INFO:teuthology.orchestra.run.smithi107.stdout:3272 read 0xad41a00 thru 0xad4f866 (0xde67 bytes) 2024-09-11T09:29:33.448 INFO:teuthology.orchestra.run.smithi107.stdout:3273 read 0x1b8be00 thru 0x1b9675f (0xa960 bytes) 2024-09-11T09:29:33.449 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:29:33.449 INFO:teuthology.orchestra.run.smithi107.stdout:3283 write 0xf0200 thru 0xf353a (0x333b bytes) 2024-09-11T09:29:33.449 INFO:teuthology.orchestra.run.smithi107.stdout:3285 read 0x94f2800 thru 0x950060e (0xde0f bytes) 2024-09-11T09:29:33.449 INFO:teuthology.orchestra.run.smithi107.stdout:3291 read 0xa980e00 thru 0xa9816a1 (0x8a2 bytes) 2024-09-11T09:29:33.449 INFO:teuthology.orchestra.run.smithi107.stdout:3292 read 0x9e59a00 thru 0x9e655c1 (0xbbc2 bytes) 2024-09-11T09:29:33.450 INFO:teuthology.orchestra.run.smithi107.stdout:3293 write 0x116a000 thru 0x116dd7e (0x3d7f bytes) 2024-09-11T09:29:33.450 INFO:teuthology.orchestra.run.smithi107.stdout:3295 read 0x7372400 thru 0x737b833 (0x9434 bytes) 2024-09-11T09:29:33.460 INFO:teuthology.orchestra.run.smithi107.stdout:3296 trunc from 0xaf08400 to 0x25c4a00 2024-09-11T09:29:33.514 INFO:teuthology.orchestra.run.smithi107.stdout:3297 read 0x20c4c00 thru 0x20ccb74 (0x7f75 bytes) 2024-09-11T09:29:33.515 INFO:teuthology.orchestra.run.smithi107.stdout:3299 write 0xa1b200 thru 0xa24287 (0x9088 bytes) 2024-09-11T09:29:33.516 INFO:teuthology.orchestra.run.smithi107.stdout:3300 read 0x21c5c00 thru 0x21c952d (0x392e bytes) 2024-09-11T09:29:33.539 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:33.540+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:33.540 INFO:teuthology.orchestra.run.smithi107.stdout:3301 write 0xdbb0c00 thru 0xdbb5fcf (0x53d0 bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3302 read 0x7ecda00 thru 0x7ecf91a (0x1f1b bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3303 write 0xb360e00 thru 0xb3681cc (0x73cd bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3304 write 0x73a1800 thru 0x73abafc (0xa2fd bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3305 write 0xa837600 thru 0xa84174d (0xa14e bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3306 read 0x3631000 thru 0x3635bda (0x4bdb bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3307 read 0xbf25e00 thru 0xbf2f323 (0x9524 bytes) 2024-09-11T09:29:33.542 INFO:teuthology.orchestra.run.smithi107.stdout:3309 read 0x36d0600 thru 0x36dbfee (0xb9ef bytes) 2024-09-11T09:29:33.547 INFO:teuthology.orchestra.run.smithi107.stdout:3312 trunc from 0xdbb6000 to 0x9161800 2024-09-11T09:29:33.548 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.877368939, 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.549 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:33.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:33.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1715 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.555 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.672145274999998, 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022344029000000001, 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1715 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:33.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.96e-07 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876407+0000", 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.871e-06 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876414+0000", 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4679999999999997e-06 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.558 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876419+0000", 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1939999999999997e-06 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898563+0000", 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022144646 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898643+0000", 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9900000000000004e-05 2024-09-11T09:29:33.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898749+0000", 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105554 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1716 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.672125013999999, 2024-09-11T09:29:33.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022552065999999999, 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1716 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:33.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876426+0000", 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.77e-07 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876432+0000", 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0079999999999999e-06 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876438+0000", 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4999999999999996e-06 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876443+0000", 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.259e-06 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898781+0000", 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022337308 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898875+0000", 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4388000000000005e-05 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898977+0000", 2024-09-11T09:29:33.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010222599999999999 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1717 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.672100108999999, 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022753961, 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.565 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1717 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876451+0000", 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876452+0000", 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.905e-06 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876459+0000", 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4980000000000003e-06 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876464+0000", 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1909999999999999e-06 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899008+0000", 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022544281999999999 2024-09-11T09:29:33.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899093+0000", 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4505000000000005e-05 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899204+0000", 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111049 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1718 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.672079445000001, 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02296196, 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1718 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.570 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:33.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876473+0000", 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9259999999999999e-06 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876480+0000", 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876485+0000", 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2159999999999997e-06 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899235+0000", 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022750631 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899321+0000", 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5958999999999995e-05 2024-09-11T09:29:33.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899433+0000", 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011126899999999999 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1719 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 262144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.672039028, 2024-09-11T09:29:33.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023149355, 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1719 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:33.575 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876512+0000", 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.44e-07 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876514+0000", 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876522+0000", 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5000000000000002e-06 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876527+0000", 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.587e-06 2024-09-11T09:29:33.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899467+0000", 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022939975000000001 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899552+0000", 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5131999999999997e-05 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.578 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899660+0000", 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108137 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1720 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.672015437999999, 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023387654000000001, 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.579 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1720 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.63e-07 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876537+0000", 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1040000000000001e-06 2024-09-11T09:29:33.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876544+0000", 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7800000000000003e-06 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876549+0000", 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2549999999999997e-06 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.582 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899706+0000", 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023156708000000002 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899801+0000", 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4826000000000003e-05 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899922+0000", 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121518 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.583 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1760 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 12288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671994546000001, 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023618561999999999, 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.584 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1760 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:33.585 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876558+0000", 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1670000000000002e-06 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.586 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876566+0000", 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3810000000000001e-06 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876571+0000", 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3029999999999999e-06 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899963+0000", 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023392445000000001 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900061+0000", 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7495000000000001e-05 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900174+0000", 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113391 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1761 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671972746000002, 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023824222999999999, 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1761 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.589 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876578+0000", 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876580+0000", 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9539999999999998e-06 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876586+0000", 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876591+0000", 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900220+0000", 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023628546 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900317+0000", 2024-09-11T09:29:33.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7168000000000002e-05 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900402+0000", 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4701000000000001e-05 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1762 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 57344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.593 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671952490999999, 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024033523000000001, 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1762 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:33.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876600+0000", 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.052e-06 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876607+0000", 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876612+0000", 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9259999999999999e-06 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900448+0000", 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023836570000000001 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900534+0000", 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5314000000000003e-05 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900631+0000", 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7676000000000005e-05 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1763 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 79872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:33.598 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671932065, 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024166084000000001, 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1763 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876619+0000", 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6400000000000003e-07 2024-09-11T09:29:33.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876621+0000", 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0949999999999998e-06 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876629+0000", 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8380000000000008e-06 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876635+0000", 2024-09-11T09:29:33.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3199999999999999e-06 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900675+0000", 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024040611 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900724+0000", 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8097999999999999e-05 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900784+0000", 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0658000000000001e-05 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1764 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671908827999999, 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024356692999999999, 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.603 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1764 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:33.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876642+0000", 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876644+0000", 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.041e-06 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876651+0000", 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6309999999999997e-06 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876656+0000", 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999999e-06 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900814+0000", 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024158129 2024-09-11T09:29:33.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900906+0000", 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2618e-05 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900998+0000", 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1624000000000002e-05 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1765 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671888098, 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024574166000000001, 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.608 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1765 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.39e-07 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876664+0000", 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9520000000000001e-06 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876671+0000", 2024-09-11T09:29:33.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3670000000000002e-06 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876676+0000", 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901029+0000", 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024353631000000001 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901109+0000", 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9951999999999996e-05 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901236+0000", 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012697000000000001 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.612 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671868309000001, 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.613 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:33.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671848077, 2024-09-11T09:29:33.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:33.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.621 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671821950999998, 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.622 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.623 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:33.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.627 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671799665999998, 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:33.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:33.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:33.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.632 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671776225999999, 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:33.636 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671756379000001, 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.637 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:33.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.671724002000001, 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:33.642 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:33.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:33.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:33.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:33.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.647 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:33.647 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:33.647 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:33.648 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:33.648+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:33.649 INFO:teuthology.orchestra.run.smithi107.stdout:3316 read 0x332a800 thru 0x33382c4 (0xdac5 bytes) 2024-09-11T09:29:33.649 INFO:teuthology.orchestra.run.smithi107.stdout:3317 read 0x3ad400 thru 0x3b3e00 (0x6a01 bytes) 2024-09-11T09:29:33.649 INFO:teuthology.orchestra.run.smithi107.stdout:3318 trunc from 0x9161800 to 0x58a200 2024-09-11T09:29:33.649 INFO:teuthology.orchestra.run.smithi107.stdout:3320 write 0x8ff3000 thru 0x8ff64f8 (0x34f9 bytes) 2024-09-11T09:29:33.649 INFO:teuthology.orchestra.run.smithi107.stdout:3321 read 0x238ee00 thru 0x239945f (0xa660 bytes) 2024-09-11T09:29:33.650 INFO:teuthology.orchestra.run.smithi107.stdout:3322 trunc from 0x8ff6600 to 0x1fb6400 2024-09-11T09:29:33.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:33.660 INFO:teuthology.orchestra.run.smithi107.stdout:3323 trunc from 0x1fb6400 to 0x9e2fa00 2024-09-11T09:29:33.675 INFO:teuthology.orchestra.run.smithi107.stdout:3324 read 0x4384400 thru 0x4391b6c (0xd76d bytes) 2024-09-11T09:29:33.675 INFO:teuthology.orchestra.run.smithi107.stdout:3325 write 0x57e3200 thru 0x57eba37 (0x8838 bytes) 2024-09-11T09:29:33.681 INFO:teuthology.orchestra.run.smithi107.stdout:3329 trunc from 0x9e2fa00 to 0xe9d8200 2024-09-11T09:29:33.693 INFO:teuthology.orchestra.run.smithi107.stdout:3330 write 0xbc53a00 thru 0xbc55da5 (0x23a6 bytes) 2024-09-11T09:29:33.694 INFO:teuthology.orchestra.run.smithi107.stdout:3333 write 0xc342200 thru 0xc347171 (0x4f72 bytes) 2024-09-11T09:29:33.705 INFO:teuthology.orchestra.run.smithi107.stdout:3334 write 0xeb0d000 thru 0xeb1698e (0x998f bytes) 2024-09-11T09:29:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:3336 read 0xb03b400 thru 0xb048cea (0xd8eb bytes) 2024-09-11T09:29:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:3339 read 0x3ef2200 thru 0x3efb4d6 (0x92d7 bytes) 2024-09-11T09:29:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:3342 write 0xc981600 thru 0xc98b346 (0x9d47 bytes) 2024-09-11T09:29:33.706 INFO:teuthology.orchestra.run.smithi107.stdout:3343 punch from 0xf80c00 to 0xf8dc00, (0xd000 bytes) 2024-09-11T09:29:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:3344 write 0x3fdc400 thru 0x3fdd539 (0x113a bytes) 2024-09-11T09:29:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:3346 write 0x53e8400 thru 0x53f38b0 (0xb4b1 bytes) 2024-09-11T09:29:33.708 INFO:teuthology.orchestra.run.smithi107.stdout:3347 read 0x588d200 thru 0x58974c1 (0xa2c2 bytes) 2024-09-11T09:29:33.709 INFO:teuthology.orchestra.run.smithi107.stdout:3349 read 0xb0c9000 thru 0xb0ca5c7 (0x15c8 bytes) 2024-09-11T09:29:33.709 INFO:teuthology.orchestra.run.smithi107.stdout:3350 read 0x812b200 thru 0x812b597 (0x398 bytes) 2024-09-11T09:29:33.709 INFO:teuthology.orchestra.run.smithi107.stdout:3352 punch from 0xdd6be00 to 0xdd73600, (0x7800 bytes) 2024-09-11T09:29:33.709 INFO:teuthology.orchestra.run.smithi107.stdout:3357 punch from 0x3847600 to 0x384c600, (0x5000 bytes) 2024-09-11T09:29:33.710 INFO:teuthology.orchestra.run.smithi107.stdout:3358 write 0xe1bf000 thru 0xe1ce504 (0xf505 bytes) 2024-09-11T09:29:33.710 INFO:teuthology.orchestra.run.smithi107.stdout:3359 read 0xe2fda00 thru 0xe2fea9a (0x109b bytes) 2024-09-11T09:29:33.710 INFO:teuthology.orchestra.run.smithi107.stdout:3360 read 0xc88c200 thru 0xc89a681 (0xe482 bytes) 2024-09-11T09:29:33.712 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:29:33.713 INFO:teuthology.orchestra.run.smithi107.stdout:3361 write 0xab65c00 thru 0xab740be (0xe4bf bytes) 2024-09-11T09:29:33.714 INFO:teuthology.orchestra.run.smithi107.stdout:3362 read 0xdf80600 thru 0xdf841a4 (0x3ba5 bytes) 2024-09-11T09:29:33.714 INFO:teuthology.orchestra.run.smithi107.stdout:3364 write 0x4c1a400 thru 0x4c1e633 (0x4234 bytes) 2024-09-11T09:29:33.714 INFO:teuthology.orchestra.run.smithi107.stdout:3365 trunc from 0xeb16a00 to 0xeab6400 2024-09-11T09:29:33.734 INFO:teuthology.orchestra.run.smithi107.stdout:3367 write 0xce86a00 thru 0xce9461a (0xdc1b bytes) 2024-09-11T09:29:33.735 INFO:teuthology.orchestra.run.smithi107.stdout:3369 write 0xe4e5a00 thru 0xe4ef0f3 (0x96f4 bytes) 2024-09-11T09:29:33.739 INFO:teuthology.orchestra.run.smithi107.stdout:3373 read 0x2f55a00 thru 0x2f5a0dc (0x46dd bytes) 2024-09-11T09:29:33.739 INFO:teuthology.orchestra.run.smithi107.stdout:3374 write 0x6883000 thru 0x6891e3d (0xee3e bytes) 2024-09-11T09:29:33.740 INFO:teuthology.orchestra.run.smithi107.stdout:3375 read 0xd021200 thru 0xd02ccf0 (0xbaf1 bytes) 2024-09-11T09:29:33.740 INFO:teuthology.orchestra.run.smithi107.stdout:3376 write 0xa958a00 thru 0xa9601cf (0x77d0 bytes) 2024-09-11T09:29:33.740 INFO:teuthology.orchestra.run.smithi107.stdout:3377 punch from 0x5a82200 to 0x5a91400, (0xf200 bytes) 2024-09-11T09:29:33.747 INFO:teuthology.orchestra.run.smithi107.stdout:3379 trunc from 0xeab6400 to 0x95bb600 2024-09-11T09:29:33.748 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:33.749+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:33.764 INFO:teuthology.orchestra.run.smithi107.stdout:3382 punch from 0x5b71800 to 0x5b7a400, (0x8c00 bytes) 2024-09-11T09:29:33.771 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:33.771 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:33.771 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:33.771 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:33.771 INFO:teuthology.orchestra.run.smithi107.stdout:3389 read 0x4285800 thru 0x428b8f4 (0x60f5 bytes) 2024-09-11T09:29:33.771 INFO:teuthology.orchestra.run.smithi107.stdout:3390 read 0xbbea00 thru 0xbc87fa (0x9dfb bytes) 2024-09-11T09:29:33.772 INFO:teuthology.orchestra.run.smithi107.stdout:3391 read 0x8b72800 thru 0x8b73751 (0xf52 bytes) 2024-09-11T09:29:33.772 INFO:teuthology.orchestra.run.smithi107.stdout:3392 read 0x11f4600 thru 0x11fd36c (0x8d6d bytes) 2024-09-11T09:29:33.772 INFO:teuthology.orchestra.run.smithi107.stdout:3395 punch from 0x422e000 to 0x4238e00, (0xae00 bytes) 2024-09-11T09:29:33.772 INFO:teuthology.orchestra.run.smithi107.stdout:3396 punch from 0x523a000 to 0x5247200, (0xd200 bytes) 2024-09-11T09:29:33.782 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:33.786 INFO:teuthology.orchestra.run.smithi107.stdout:3397 write 0x9ed5200 thru 0x9eda8ab (0x56ac bytes) 2024-09-11T09:29:33.786 INFO:teuthology.orchestra.run.smithi107.stdout:3400 clone 20 order 23 su 131072 sc 4 2024-09-11T09:29:33.850 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:33.850+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:33.891 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:33.892 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:33.892 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:33.892 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:33.892 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:33.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:33.951 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:33.951+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.098786296, 2024-09-11T09:29:34.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.011 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:34.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:34.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:34.015 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.098555724000001, 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:34.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.098458520000001, 2024-09-11T09:29:34.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:34.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:34.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:34.024 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.098223755999999, 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.025 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.027 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:34.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:34.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.098027739999999, 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.031 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:34.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.097759303, 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:34.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:34.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.040 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.097657553000001, 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:34.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:34.045 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.097263896000001, 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:34.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:34.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.097137651000001, 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.050 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:34.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.096928268999999, 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.055 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.056 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:34.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.09681732, 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.060 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:34.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.096561382000001, 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:34.064 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:34.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:34.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:34.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.096408853, 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.069 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.073 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.096216489, 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:34.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:34.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.095886070999999, 2024-09-11T09:29:34.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:34.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:34.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:34.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.095764518999999, 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.083 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:34.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.086 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.087 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.095609741000001, 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.088 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:34.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.092 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.095555821000001, 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:34.096 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.095355252000001, 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.097 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:34.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.094999713, 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.102 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:34.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.106 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:34.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:34.108 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:34.109+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:34.209 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:34.209+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:34.257 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:34.258 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:34.258 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:34.258 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:34.266 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:34.309 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:34.310+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:34.372 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:34.373 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:34.373 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:34.373 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:34.373 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:34.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:34.423 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:34.423+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:34.511 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:34.512+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:34.529 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.529 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 58.070168811000002, 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:34.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:34.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:34.534 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.808649123000002, 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:34.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:34.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:34.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.540 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.807403155999999, 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:34.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:34.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.544 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:34.545 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:34.545 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:34.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:34.612 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:34.612+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:34.660 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:34.660 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:34.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:34.702 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:29:34.752 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:34.753+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:34.774 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone19 from 0x9edaa00 (overlap 0x58a200) to 0x0 2024-09-11T09:29:34.814 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:34.815+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:34.822 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:34.823 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:34.823 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:34.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:34.915 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:34.916+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:34.979 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.541959360000007, 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:34.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:34.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:34.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:34.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:34.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:34.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5081 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017684165999999, 2024-09-11T09:29:34.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043672824999999998, 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5081 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:34.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961281+0000", 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961285+0000", 2024-09-11T09:29:34.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3560000000000001e-06 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961287+0000", 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5409999999999999e-06 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004781+0000", 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043493472999999998 2024-09-11T09:29:34.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004856+0000", 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5741000000000005e-05 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004947+0000", 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1222000000000002e-05 2024-09-11T09:29:34.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5083 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017668592, 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043785219, 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:34.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5083 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961291+0000", 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961292+0000", 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961295+0000", 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0149999999999999e-06 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961297+0000", 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004977+0000", 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043679791000000003 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005021+0000", 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4295000000000002e-05 2024-09-11T09:29:34.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005075+0000", 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999998e-05 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5088 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 905216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017659044999998, 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043963918999999997, 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5088 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:34.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961301+0000", 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.13e-06 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961305+0000", 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4529999999999998e-06 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961309+0000", 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005096+0000", 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043787848999999997 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005175+0000", 2024-09-11T09:29:35.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8487999999999998e-05 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005264+0000", 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8786e-05 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5097 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1605632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017622198000002, 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044049360000000003, 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5097 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:35.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961339+0000", 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1440000000000001e-06 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961345+0000", 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7470000000000002e-06 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961350+0000", 2024-09-11T09:29:35.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209999999999996e-06 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005291+0000", 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043940405000000002 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005332+0000", 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1542999999999998e-05 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005386+0000", 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999998e-05 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5100 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017602347, 2024-09-11T09:29:35.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044229475999999997, 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5100 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:35.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961357+0000", 2024-09-11T09:29:35.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961359+0000", 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.052e-06 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961365+0000", 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961369+0000", 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6689999999999997e-06 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005422+0000", 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044052701 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005499+0000", 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6664999999999999e-05 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005586+0000", 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7230000000000003e-05 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.012 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5105 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017583766000001, 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044370891000000003, 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5105 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961377+0000", 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9120000000000001e-06 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961383+0000", 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-09-11T09:29:35.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961388+0000", 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005629+0000", 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044240743999999999 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005687+0000", 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7951000000000001e-05 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005746+0000", 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9444999999999998e-05 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5108 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1607680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017565408999999, 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581704, 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5108 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961394+0000", 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961396+0000", 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961401+0000", 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8119999999999996e-06 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961406+0000", 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4920000000000004e-06 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005789+0000", 2024-09-11T09:29:35.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383498 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005882+0000", 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2541999999999996e-05 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005975+0000", 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3110999999999997e-05 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5111 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017547364999999, 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044694581999999997, 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5111 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961412+0000", 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961414+0000", 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.996e-06 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961420+0000", 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8000000000000004e-06 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961425+0000", 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5149999999999997e-06 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006007+0000", 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581786999999998 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006051+0000", 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3998999999999997e-05 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006106+0000", 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5145999999999997e-05 2024-09-11T09:29:35.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5116 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1632256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017527960999999, 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044886862999999999, 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5116 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:35.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961433+0000", 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961439+0000", 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961444+0000", 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809999999999998e-06 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006144+0000", 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044699782 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006221+0000", 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7270999999999999e-05 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006318+0000", 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6914000000000001e-05 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5121 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1611776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017509539999999, 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045003543, 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5121 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.033 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961450+0000", 2024-09-11T09:29:35.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961452+0000", 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0030000000000001e-06 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961459+0000", 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961464+0000", 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006348+0000", 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044884563000000002 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006396+0000", 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7817e-05 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006453+0000", 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6762000000000003e-05 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5124 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017489071, 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045118734000000001, 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5124 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961472+0000", 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929999999999999e-06 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961478+0000", 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.592e-06 2024-09-11T09:29:35.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961482+0000", 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006470+0000", 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044987680000000002 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006529+0000", 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9116000000000001e-05 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006589+0000", 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9463000000000003e-05 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5129 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1624064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017470702000001, 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045244390000000002, 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5129 2024-09-11T09:29:35.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000002e-07 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961490+0000", 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961496+0000", 2024-09-11T09:29:35.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961500+0000", 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3819999999999997e-06 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006606+0000", 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045106159 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006669+0000", 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2188999999999997e-05 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006733+0000", 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3848000000000003e-05 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5134 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.047 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017435730999999, 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045344917999999998, 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5134 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961524+0000", 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961531+0000", 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961541+0000", 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0059e-05 2024-09-11T09:29:35.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961546+0000", 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006750+0000", 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045204167000000003 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006808+0000", 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8229000000000001e-05 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006868+0000", 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9953e-05 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.052 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5135 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1622016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.01740603, 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045554071000000002, 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5135 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.053 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7100000000000002e-07 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961555+0000", 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1679999999999998e-06 2024-09-11T09:29:35.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961561+0000", 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961566+0000", 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4170000000000002e-06 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006908+0000", 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045342423999999999 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007004+0000", 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5307000000000001e-05 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007107+0000", 2024-09-11T09:29:35.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103253 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5138 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1634304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017386260999999, 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045679006000000001, 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5138 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961805+0000", 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023282700000000001 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961816+0000", 2024-09-11T09:29:35.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961828+0000", 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2318e-05 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961835+0000", 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007139+0000", 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045303738000000003 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007188+0000", 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9014999999999997e-05 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007252+0000", 2024-09-11T09:29:35.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3921e-05 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5143 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1613824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.017112876999999, 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045634479999999998, 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5143 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961994+0000", 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147949 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961997+0000", 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5580000000000001e-06 2024-09-11T09:29:35.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962007+0000", 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6579999999999997e-06 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962014+0000", 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007290+0000", 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045276416 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007380+0000", 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9686999999999997e-05 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007480+0000", 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100762 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.068 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5148 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1626112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.016934986999999, 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045674845999999998, 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.069 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5148 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962412+0000", 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038829000000000001 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962417+0000", 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962427+0000", 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962434+0000", 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7379999999999997e-06 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007514+0000", 2024-09-11T09:29:35.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045080264000000002 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007600+0000", 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5970000000000005e-05 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007699+0000", 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8706e-05 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.073 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5153 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1615872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.016513576000001, 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045448919999999997, 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.074 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5153 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:35.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962446+0000", 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962448+0000", 2024-09-11T09:29:35.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962455+0000", 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962458+0000", 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007712+0000", 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045253466999999999 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007784+0000", 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1939000000000004e-05 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007894+0000", 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5156 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 24.016494517999998, 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045635835999999999, 2024-09-11T09:29:35.079 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5156 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962886+0000", 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042155799999999999 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962890+0000", 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962899+0000", 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0440000000000001e-06 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962906+0000", 2024-09-11T09:29:35.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1169999999999998e-06 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007917+0000", 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045011412000000001 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007994+0000", 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6650999999999995e-05 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:35.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.008100+0000", 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106164 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:35.084 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:35.085 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-11T09:29:35.086 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:35.086+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:35.086 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:35.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:35.226 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:35.227+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:35.250 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:35.250 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:35.250 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:35.250 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:35.259 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:35.302 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:35.303+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:35.316 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:35.316 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:35.334 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:35.334 INFO:teuthology.orchestra.run.smithi107.stdout:3402 read 0x7fe2c00 thru 0x7fe68cf (0x3cd0 bytes) 2024-09-11T09:29:35.334 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:35.335 INFO:teuthology.orchestra.run.smithi107.stdout:3403 write 0x7718000 thru 0x77223a3 (0xa3a4 bytes) 2024-09-11T09:29:35.335 INFO:teuthology.orchestra.run.smithi107.stdout:3404 write 0x326ea00 thru 0x326f29e (0x89f bytes) 2024-09-11T09:29:35.336 INFO:teuthology.orchestra.run.smithi107.stdout:3405 write 0x8b21800 thru 0x8b2deed (0xc6ee bytes) 2024-09-11T09:29:35.338 INFO:teuthology.orchestra.run.smithi107.stdout:3406 write 0x41af200 thru 0x41bbf1e (0xcd1f bytes) 2024-09-11T09:29:35.338 INFO:teuthology.orchestra.run.smithi107.stdout:3407 read 0x2f1d400 thru 0x2f240f2 (0x6cf3 bytes) 2024-09-11T09:29:35.342 INFO:teuthology.orchestra.run.smithi107.stdout:3408 trunc from 0x9edaa00 to 0x8012200 2024-09-11T09:29:35.357 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:35.357 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:35.374 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:35.374 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:35.374 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:35.382 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:35.382 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:35.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:35.391 INFO:teuthology.orchestra.run.smithi107.stdout:3409 write 0x762b800 thru 0x762bad0 (0x2d1 bytes) 2024-09-11T09:29:35.392 INFO:teuthology.orchestra.run.smithi107.stdout:3410 punch from 0x11c0e00 to 0x11c2200, (0x1400 bytes) 2024-09-11T09:29:35.398 INFO:teuthology.orchestra.run.smithi107.stdout:3412 trunc from 0x8012200 to 0xa2f0e00 2024-09-11T09:29:35.404 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:35.404 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:35.427 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:35.427+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:35.428 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:35.429 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:35.436 INFO:teuthology.orchestra.run.smithi107.stdout:3413 punch from 0x9d0f200 to 0x9d17c00, (0x8a00 bytes) 2024-09-11T09:29:35.438 INFO:teuthology.orchestra.run.smithi107.stdout:3414 write 0x9671200 thru 0x9677f32 (0x6d33 bytes) 2024-09-11T09:29:35.443 INFO:teuthology.orchestra.run.smithi107.stdout:3415 trunc from 0xa2f0e00 to 0xe842600 2024-09-11T09:29:35.456 INFO:teuthology.orchestra.run.smithi107.stdout:3416 trunc from 0xe842600 to 0xdb3ca00 2024-09-11T09:29:35.466 INFO:teuthology.orchestra.run.smithi107.stdout:3421 read 0x8350600 thru 0x835a572 (0x9f73 bytes) 2024-09-11T09:29:35.468 INFO:teuthology.orchestra.run.smithi107.stdout:3422 punch from 0x8517000 to 0x8522400, (0xb400 bytes) 2024-09-11T09:29:35.477 INFO:teuthology.orchestra.run.smithi107.stdout:3423 trunc from 0xdb3ca00 to 0x64a7000 2024-09-11T09:29:35.489 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:35.489+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:35.504 INFO:teuthology.orchestra.run.smithi107.stdout:3424 clone 21 order 25 su 524288 sc 12 2024-09-11T09:29:35.544 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:35.544 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:35.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:35.589 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:35.590+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:35.690 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:35.691+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:35.695 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:35.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:35.781 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone20 from 0x64a7000 (overlap 0x64a7000) to 0xa10a00 2024-09-11T09:29:35.791 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:35.791+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:35.880 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:35.881 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:35.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:35.893 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:35.892+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:35.993 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:35.994+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:36.041 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.041 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:36.095 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:36.095+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:36.161 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.161 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.170 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:36.195 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:36.196+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:36.213 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-11T09:29:36.270 INFO:teuthology.orchestra.run.smithi107.stdout:3425 write 0x49f9800 thru 0x4a083a2 (0xeba3 bytes) 2024-09-11T09:29:36.271 INFO:teuthology.orchestra.run.smithi107.stdout:3426 read 0x49ea000 thru 0x49f20f4 (0x80f5 bytes) 2024-09-11T09:29:36.272 INFO:teuthology.orchestra.run.smithi107.stdout:3427 read 0x5c93800 thru 0x5c9d0c3 (0x98c4 bytes) 2024-09-11T09:29:36.296 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:36.296+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:36.318 INFO:teuthology.orchestra.run.smithi107.stdout:3429 write 0x6973200 thru 0x69813eb (0xe1ec bytes) 2024-09-11T09:29:36.321 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.321 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.326 INFO:teuthology.orchestra.run.smithi107.stdout:3430 punch from 0x5f51c00 to 0x5f5ae00, (0x9200 bytes) 2024-09-11T09:29:36.327 INFO:teuthology.orchestra.run.smithi107.stdout:3431 read 0x201e400 thru 0x202de1e (0xfa1f bytes) 2024-09-11T09:29:36.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:36.338 INFO:teuthology.orchestra.run.smithi107.stdout:3432 write 0x353f000 thru 0x354afe1 (0xbfe2 bytes) 2024-09-11T09:29:36.342 INFO:teuthology.orchestra.run.smithi107.stdout:3433 read 0x922e00 thru 0x92484e (0x1a4f bytes) 2024-09-11T09:29:36.342 INFO:teuthology.orchestra.run.smithi107.stdout:3434 read 0x1bb3800 thru 0x1bb4aab (0x12ac bytes) 2024-09-11T09:29:36.380 INFO:teuthology.orchestra.run.smithi107.stdout:3435 write 0x7a31c00 thru 0x7a35d56 (0x4157 bytes) 2024-09-11T09:29:36.382 INFO:teuthology.orchestra.run.smithi107.stdout:3437 write 0xa9c800 thru 0xaa5842 (0x9043 bytes) 2024-09-11T09:29:36.397 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:36.398+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:36.404 INFO:teuthology.orchestra.run.smithi107.stdout:3439 write 0xa4bb800 thru 0xa4c7f74 (0xc775 bytes) 2024-09-11T09:29:36.429 INFO:teuthology.orchestra.run.smithi107.stdout:3443 write 0xd893800 thru 0xd899e95 (0x6696 bytes) 2024-09-11T09:29:36.430 INFO:teuthology.orchestra.run.smithi107.stdout:3447 read 0x453a00 thru 0x45b63d (0x7c3e bytes) 2024-09-11T09:29:36.431 INFO:teuthology.orchestra.run.smithi107.stdout:3449 read 0x3718c00 thru 0x371f24a (0x664b bytes) 2024-09-11T09:29:36.432 INFO:teuthology.orchestra.run.smithi107.stdout:3451 write 0x7c15200 thru 0x7c20033 (0xae34 bytes) 2024-09-11T09:29:36.432 INFO:teuthology.orchestra.run.smithi107.stdout:3454 punch from 0x62e9800 to 0x62f7000, (0xd800 bytes) 2024-09-11T09:29:36.433 INFO:teuthology.orchestra.run.smithi107.stdout:3457 read 0x92aae00 thru 0x92ab749 (0x94a bytes) 2024-09-11T09:29:36.443 INFO:teuthology.orchestra.run.smithi107.stdout:3459 write 0x6164a00 thru 0x616e2a9 (0x98aa bytes) 2024-09-11T09:29:36.464 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.464 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.475 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:36.499 INFO:teuthology.orchestra.run.smithi107.stdout:3461 write 0xdaa1c00 thru 0xdaa668b (0x4a8c bytes) 2024-09-11T09:29:36.505 INFO:teuthology.orchestra.run.smithi107.stdout:3462 trunc from 0xdaa6800 to 0x99d4c00 2024-09-11T09:29:36.518 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:36.519+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:36.552 INFO:teuthology.orchestra.run.smithi107.stdout:3464 read 0x603c600 thru 0x604b66c (0xf06d bytes) 2024-09-11T09:29:36.559 INFO:teuthology.orchestra.run.smithi107.stdout:3465 read 0xf1f600 thru 0xf24965 (0x5366 bytes) 2024-09-11T09:29:36.560 INFO:teuthology.orchestra.run.smithi107.stdout:3468 punch from 0x6277400 to 0x6282400, (0xb000 bytes) 2024-09-11T09:29:36.560 INFO:teuthology.orchestra.run.smithi107.stdout:3469 read 0x3a38a00 thru 0x3a3a576 (0x1b77 bytes) 2024-09-11T09:29:36.561 INFO:teuthology.orchestra.run.smithi107.stdout:3471 punch from 0x30de400 to 0x30e2400, (0x4000 bytes) 2024-09-11T09:29:36.561 INFO:teuthology.orchestra.run.smithi107.stdout:3472 read 0x47a7800 thru 0x47b56a5 (0xdea6 bytes) 2024-09-11T09:29:36.561 INFO:teuthology.orchestra.run.smithi107.stdout:3475 write 0x5bf6200 thru 0x5bf861b (0x241c bytes) 2024-09-11T09:29:36.562 INFO:teuthology.orchestra.run.smithi107.stdout:3476 flatten 2024-09-11T09:29:36.573 INFO:teuthology.orchestra.run.smithi107.stdout:3477 read 0x7640200 thru 0x7646b37 (0x6938 bytes) 2024-09-11T09:29:36.574 INFO:teuthology.orchestra.run.smithi107.stdout:3480 read 0x18f0000 thru 0x18fd28a (0xd28b bytes) 2024-09-11T09:29:36.575 INFO:teuthology.orchestra.run.smithi107.stdout:3481 punch from 0x1503a00 to 0x1507800, (0x3e00 bytes) 2024-09-11T09:29:36.592 INFO:teuthology.orchestra.run.smithi107.stdout:3483 write 0xa073400 thru 0xa076d97 (0x3998 bytes) 2024-09-11T09:29:36.593 INFO:teuthology.orchestra.run.smithi107.stdout:3484 write 0x80d3000 thru 0x80d9fe4 (0x6fe5 bytes) 2024-09-11T09:29:36.594 INFO:teuthology.orchestra.run.smithi107.stdout:3485 read 0x9ade200 thru 0x9ae140d (0x320e bytes) 2024-09-11T09:29:36.596 INFO:teuthology.orchestra.run.smithi107.stdout:3486 trunc from 0xa076e00 to 0x8976a00 2024-09-11T09:29:36.599 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.599 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.599 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:36.599+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:36.609 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:36.617 INFO:teuthology.orchestra.run.smithi107.stdout:3488 punch from 0x209f400 to 0x20a1000, (0x1c00 bytes) 2024-09-11T09:29:36.617 INFO:teuthology.orchestra.run.smithi107.stdout:3489 punch from 0x7c64600 to 0x7c69e00, (0x5800 bytes) 2024-09-11T09:29:36.618 INFO:teuthology.orchestra.run.smithi107.stdout:3490 write 0x263d000 thru 0x264628d (0x928e bytes) 2024-09-11T09:29:36.618 INFO:teuthology.orchestra.run.smithi107.stdout:3491 write 0x7aac200 thru 0x7ab45c3 (0x83c4 bytes) 2024-09-11T09:29:36.619 INFO:teuthology.orchestra.run.smithi107.stdout:3492 read 0x286d200 thru 0x2872aac (0x58ad bytes) 2024-09-11T09:29:36.635 INFO:teuthology.orchestra.run.smithi107.stdout:3495 write 0xd6ad200 thru 0xd6b5801 (0x8602 bytes) 2024-09-11T09:29:36.653 INFO:teuthology.orchestra.run.smithi107.stdout:3496 trunc from 0xd6b5a00 to 0x2748400 2024-09-11T09:29:36.684 INFO:teuthology.orchestra.run.smithi107.stdout:3500 read 0x14ecc00 thru 0x14f4345 (0x7746 bytes) 2024-09-11T09:29:36.686 INFO:teuthology.orchestra.run.smithi107.stdout:3504 punch from 0x2337600 to 0x2344a00, (0xd400 bytes) 2024-09-11T09:29:36.700 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:36.701+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:36.702 INFO:teuthology.orchestra.run.smithi107.stdout:3505 write 0x81cf000 thru 0x81dc186 (0xd187 bytes) 2024-09-11T09:29:36.704 INFO:teuthology.orchestra.run.smithi107.stdout:3506 read 0x658ce00 thru 0x6597560 (0xa761 bytes) 2024-09-11T09:29:36.704 INFO:teuthology.orchestra.run.smithi107.stdout:3508 punch from 0x61c7c00 to 0x61c9200, (0x1600 bytes) 2024-09-11T09:29:36.710 INFO:teuthology.orchestra.run.smithi107.stdout:3511 trunc from 0x81dc200 to 0x2e9d400 2024-09-11T09:29:36.716 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:29:36.725 INFO:teuthology.orchestra.run.smithi107.stdout:3512 read 0xaf2600 thru 0xafb7e2 (0x91e3 bytes) 2024-09-11T09:29:36.726 INFO:teuthology.orchestra.run.smithi107.stdout:3513 punch from 0x450600 to 0x458a00, (0x8400 bytes) 2024-09-11T09:29:36.726 INFO:teuthology.orchestra.run.smithi107.stdout:3514 write 0x1f6c00 thru 0x1fdd8d (0x718e bytes) 2024-09-11T09:29:36.750 INFO:teuthology.orchestra.run.smithi107.stdout:3517 write 0xc387800 thru 0xc38d75d (0x5f5e bytes) 2024-09-11T09:29:36.752 INFO:teuthology.orchestra.run.smithi107.stdout:3518 write 0x6a1a800 thru 0x6a24f02 (0xa703 bytes) 2024-09-11T09:29:36.752 INFO:teuthology.orchestra.run.smithi107.stdout:3519 read 0x42e9a00 thru 0x42ef73e (0x5d3f bytes) 2024-09-11T09:29:36.753 INFO:teuthology.orchestra.run.smithi107.stdout:3521 read 0x6886000 thru 0x688b8a5 (0x58a6 bytes) 2024-09-11T09:29:36.755 INFO:teuthology.orchestra.run.smithi107.stdout:3522 trunc from 0xc38d800 to 0xa874a00 2024-09-11T09:29:36.776 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.776 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.783 INFO:teuthology.orchestra.run.smithi107.stdout:3524 trunc from 0xa874a00 to 0x69e800 2024-09-11T09:29:36.785 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:36.802 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:36.802+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:36.823 INFO:teuthology.orchestra.run.smithi107.stdout:3528 trunc from 0x69e800 to 0x890d000 2024-09-11T09:29:36.832 INFO:teuthology.orchestra.run.smithi107.stdout:3529 write 0x4393c00 thru 0x43a0f16 (0xd317 bytes) 2024-09-11T09:29:36.859 INFO:teuthology.orchestra.run.smithi107.stdout:3532 write 0xba7ca00 thru 0xba8c857 (0xfe58 bytes) 2024-09-11T09:29:36.860 INFO:teuthology.orchestra.run.smithi107.stdout:3533 read 0x67cb400 thru 0x67db07b (0xfc7c bytes) 2024-09-11T09:29:36.863 INFO:teuthology.orchestra.run.smithi107.stdout:3534 trunc from 0xba8ca00 to 0x9cf3200 2024-09-11T09:29:36.890 INFO:teuthology.orchestra.run.smithi107.stdout:3539 write 0x3dd400 thru 0x3e7a77 (0xa678 bytes) 2024-09-11T09:29:36.892 INFO:teuthology.orchestra.run.smithi107.stdout:3540 read 0x7ea0800 thru 0x7eae771 (0xdf72 bytes) 2024-09-11T09:29:36.893 INFO:teuthology.orchestra.run.smithi107.stdout:3542 read 0x6cd9a00 thru 0x6cdb61d (0x1c1e bytes) 2024-09-11T09:29:36.901 INFO:teuthology.orchestra.run.smithi107.stdout:3547 trunc from 0x9cf3200 to 0x28fd400 2024-09-11T09:29:36.902 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:36.903+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:36.912 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:36.912 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:36.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:36.961 INFO:teuthology.orchestra.run.smithi107.stdout:3549 write 0xe32b600 thru 0xe33b1c8 (0xfbc9 bytes) 2024-09-11T09:29:36.961 INFO:teuthology.orchestra.run.smithi107.stdout:3550 punch from 0x85c8200 to 0x85cc200, (0x4000 bytes) 2024-09-11T09:29:36.961 INFO:teuthology.orchestra.run.smithi107.stdout:3551 punch from 0xb522200 to 0xb530800, (0xe600 bytes) 2024-09-11T09:29:36.974 INFO:teuthology.orchestra.run.smithi107.stdout:3552 trunc from 0xe33b200 to 0x2d09a00 2024-09-11T09:29:37.003 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:37.004+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:37.018 INFO:teuthology.orchestra.run.smithi107.stdout:3554 read 0x1128e00 thru 0x112d65f (0x4860 bytes) 2024-09-11T09:29:37.018 INFO:teuthology.orchestra.run.smithi107.stdout:3559 read 0x1747c00 thru 0x174d7f9 (0x5bfa bytes) 2024-09-11T09:29:37.018 INFO:teuthology.orchestra.run.smithi107.stdout:3560 read 0x2c9ae00 thru 0x2c9e899 (0x3a9a bytes) 2024-09-11T09:29:37.022 INFO:teuthology.orchestra.run.smithi107.stdout:3561 trunc from 0x2d09a00 to 0x5f44400 2024-09-11T09:29:37.038 INFO:teuthology.orchestra.run.smithi107.stdout:3562 punch from 0x2fb8a00 to 0x2fb8c00, (0x200 bytes) 2024-09-11T09:29:37.040 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:37.040 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:37.042 INFO:teuthology.orchestra.run.smithi107.stdout:3563 trunc from 0x5f44400 to 0x99efe00 2024-09-11T09:29:37.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:37.055 INFO:teuthology.orchestra.run.smithi107.stdout:3564 read 0x19b3e00 thru 0x19bc1bd (0x83be bytes) 2024-09-11T09:29:37.057 INFO:teuthology.orchestra.run.smithi107.stdout:3565 read 0x1b12600 thru 0x1b15486 (0x2e87 bytes) 2024-09-11T09:29:37.057 INFO:teuthology.orchestra.run.smithi107.stdout:3566 read 0x3579a00 thru 0x3586bf1 (0xd1f2 bytes) 2024-09-11T09:29:37.057 INFO:teuthology.orchestra.run.smithi107.stdout:3567 write 0x626c200 thru 0x6274329 (0x812a bytes) 2024-09-11T09:29:37.058 INFO:teuthology.orchestra.run.smithi107.stdout:3568 read 0x1ef6c00 thru 0x1f03ee7 (0xd2e8 bytes) 2024-09-11T09:29:37.062 INFO:teuthology.orchestra.run.smithi107.stdout:3569 trunc from 0x99efe00 to 0xdae4800 2024-09-11T09:29:37.080 INFO:teuthology.orchestra.run.smithi107.stdout:3570 write 0x4189200 thru 0x419871b (0xf51c bytes) 2024-09-11T09:29:37.080 INFO:teuthology.orchestra.run.smithi107.stdout:3574 write 0x6063400 thru 0x60692c7 (0x5ec8 bytes) 2024-09-11T09:29:37.081 INFO:teuthology.orchestra.run.smithi107.stdout:3578 clone 22 order 22 su 2097152 sc 11 2024-09-11T09:29:37.104 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:37.104+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:37.199 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:37.199 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:37.204 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:37.205+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:37.208 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:37.305 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:37.305+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:37.355 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:37.355 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:37.364 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:37.405 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:37.406+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:37.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:37.479 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:37.479 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:37.488 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:37.532 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:37.533+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:37.563 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:37.563 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:37.580 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:37.581 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:37.591 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:37.591 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:37.596 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:37.596 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:37.600 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:37.613 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:37.614 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:37.631 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:37.631 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:37.641 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:37.641+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:37.648 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:37.648 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:37.708 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:37.708+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:37.713 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:29:37.714 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:37.714 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:37.714 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-11T09:29:37.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-11T09:29:37.747 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 67.309993177999999, 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5081 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785717984000001, 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043672824999999998, 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5081 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961281+0000", 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961285+0000", 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3560000000000001e-06 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961287+0000", 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5409999999999999e-06 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004781+0000", 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043493472999999998 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004856+0000", 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5741000000000005e-05 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004947+0000", 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1222000000000002e-05 2024-09-11T09:29:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5083 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785702409999999, 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043785219, 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5083 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961291+0000", 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961292+0000", 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961295+0000", 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0149999999999999e-06 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961297+0000", 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004977+0000", 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043679791000000003 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005021+0000", 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4295000000000002e-05 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005075+0000", 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999998e-05 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5088 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 905216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785692863000001, 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043963918999999997, 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5088 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961301+0000", 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.13e-06 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961305+0000", 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4529999999999998e-06 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961309+0000", 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005096+0000", 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043787848999999997 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005175+0000", 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8487999999999998e-05 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005264+0000", 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8786e-05 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5097 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1605632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785656016000001, 2024-09-11T09:29:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044049360000000003, 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5097 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961339+0000", 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1440000000000001e-06 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961345+0000", 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7470000000000002e-06 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961350+0000", 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209999999999996e-06 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005291+0000", 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043940405000000002 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005332+0000", 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1542999999999998e-05 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005386+0000", 2024-09-11T09:29:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999998e-05 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5100 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785636165, 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044229475999999997, 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5100 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961357+0000", 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961359+0000", 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.052e-06 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961365+0000", 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961369+0000", 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6689999999999997e-06 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005422+0000", 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044052701 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005499+0000", 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6664999999999999e-05 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005586+0000", 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7230000000000003e-05 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5105 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785617584000001, 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044370891000000003, 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5105 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961377+0000", 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9120000000000001e-06 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961383+0000", 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-09-11T09:29:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961388+0000", 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005629+0000", 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044240743999999999 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005687+0000", 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7951000000000001e-05 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005746+0000", 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9444999999999998e-05 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5108 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1607680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785599226999999, 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581704, 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5108 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961394+0000", 2024-09-11T09:29:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961396+0000", 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961401+0000", 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8119999999999996e-06 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961406+0000", 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4920000000000004e-06 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005789+0000", 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383498 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005882+0000", 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2541999999999996e-05 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005975+0000", 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3110999999999997e-05 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5111 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785581183000001, 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044694581999999997, 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5111 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961412+0000", 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961414+0000", 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.996e-06 2024-09-11T09:29:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961420+0000", 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8000000000000004e-06 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961425+0000", 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5149999999999997e-06 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006007+0000", 2024-09-11T09:29:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581786999999998 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006051+0000", 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3998999999999997e-05 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006106+0000", 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5145999999999997e-05 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5116 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1632256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785561778999998, 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044886862999999999, 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5116 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961433+0000", 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961439+0000", 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961444+0000", 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809999999999998e-06 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006144+0000", 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044699782 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006221+0000", 2024-09-11T09:29:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7270999999999999e-05 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006318+0000", 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6914000000000001e-05 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5121 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1611776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785543357999998, 2024-09-11T09:29:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045003543, 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5121 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961450+0000", 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961452+0000", 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0030000000000001e-06 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961459+0000", 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961464+0000", 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006348+0000", 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044884563000000002 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006396+0000", 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7817e-05 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006453+0000", 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6762000000000003e-05 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5124 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785522888999999, 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045118734000000001, 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5124 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961472+0000", 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929999999999999e-06 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961478+0000", 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.592e-06 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961482+0000", 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-09-11T09:29:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006470+0000", 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044987680000000002 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006529+0000", 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9116000000000001e-05 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006589+0000", 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9463000000000003e-05 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5129 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1624064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.78550452, 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045244390000000002, 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5129 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000002e-07 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961490+0000", 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961496+0000", 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961500+0000", 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3819999999999997e-06 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006606+0000", 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045106159 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006669+0000", 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2188999999999997e-05 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006733+0000", 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3848000000000003e-05 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5134 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785469548999998, 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045344917999999998, 2024-09-11T09:29:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5134 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961524+0000", 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961531+0000", 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961541+0000", 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0059e-05 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961546+0000", 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006750+0000", 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045204167000000003 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006808+0000", 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8229000000000001e-05 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006868+0000", 2024-09-11T09:29:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9953e-05 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5135 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1622016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785439847999999, 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045554071000000002, 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5135 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7100000000000002e-07 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961555+0000", 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1679999999999998e-06 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961561+0000", 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961566+0000", 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4170000000000002e-06 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006908+0000", 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045342423999999999 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007004+0000", 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5307000000000001e-05 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007107+0000", 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103253 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5138 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1634304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785420079000001, 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045679006000000001, 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5138 2024-09-11T09:29:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961805+0000", 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023282700000000001 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961816+0000", 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961828+0000", 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2318e-05 2024-09-11T09:29:37.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961835+0000", 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007139+0000", 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045303738000000003 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007188+0000", 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9014999999999997e-05 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007252+0000", 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3921e-05 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5143 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1613824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.826 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.785146695000002, 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045634479999999998, 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5143 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961994+0000", 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147949 2024-09-11T09:29:37.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961997+0000", 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5580000000000001e-06 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962007+0000", 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6579999999999997e-06 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962014+0000", 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007290+0000", 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045276416 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007380+0000", 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9686999999999997e-05 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007480+0000", 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100762 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5148 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1626112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.784968804999998, 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045674845999999998, 2024-09-11T09:29:37.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5148 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962412+0000", 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038829000000000001 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962417+0000", 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962427+0000", 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962434+0000", 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7379999999999997e-06 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007514+0000", 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045080264000000002 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007600+0000", 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5970000000000005e-05 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007699+0000", 2024-09-11T09:29:37.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8706e-05 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5153 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1615872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.784547394000001, 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045448919999999997, 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5153 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:37.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962446+0000", 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962448+0000", 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962455+0000", 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962458+0000", 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007712+0000", 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045253466999999999 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007784+0000", 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1939000000000004e-05 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007894+0000", 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5156 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 26.784528336000001, 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045635835999999999, 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5156 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962886+0000", 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042155799999999999 2024-09-11T09:29:37.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962890+0000", 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962899+0000", 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0440000000000001e-06 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962906+0000", 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1169999999999998e-06 2024-09-11T09:29:37.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007917+0000", 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045011412000000001 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007994+0000", 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6650999999999995e-05 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:37.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.008100+0000", 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106164 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:37.845 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:37.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:37.847 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:37.847+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:37.947 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:37.948+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:37.993 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:37.993 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:37.993 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:38.002 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:38.021 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-11T09:29:38.022+0000 7fef02752640 -1 mon.a@0(leader).osd e61 definitely_dead 0 2024-09-11T09:29:38.048 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:38.049+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:38.108 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:38.108 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:38.108 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:38.117 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:38.157 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:38.158+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:38.180 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-11T09:29:38.251 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:38.252+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:38.266 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899613686999999, 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:38.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.271 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899576302, 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:38.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:38.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899556984, 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:38.276 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:38.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:38.280 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899545738, 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899503832999997, 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899486429, 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899468534, 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899448354, 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899427369999998, 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899400272999998, 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899380985999997, 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.899176625999999, 2024-09-11T09:29:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.898642870000003, 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.898593976999997, 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.898545888999998, 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.898507850999998, 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.898284009000001, 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.898106906999999, 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.897961352000003, 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 34.897839965999999, 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.361 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:38.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:38.363 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:38.362+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:38.363 INFO:teuthology.orchestra.run.smithi107.stdout:3579 write 0x1d75400 thru 0x1d7c5bb (0x71bc bytes) 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3581 write 0xcc38000 thru 0xcc45f37 (0xdf38 bytes) 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3582 write 0x167200 thru 0x168f83 (0x1d84 bytes) 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3583 write 0x5f6d600 thru 0x5f72170 (0x4b71 bytes) 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3584 write 0xd98e000 thru 0xd993096 (0x5097 bytes) 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3585 read 0xd7b4000 thru 0xd7b8de4 (0x4de5 bytes) 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3587 trunc from 0xdae4800 to 0x98b2a00 2024-09-11T09:29:38.364 INFO:teuthology.orchestra.run.smithi107.stdout:3589 trunc from 0x98b2a00 to 0x13f5600 2024-09-11T09:29:38.402 INFO:teuthology.orchestra.run.smithi107.stdout:3590 write 0x6457800 thru 0x6466815 (0xf016 bytes) 2024-09-11T09:29:38.424 INFO:teuthology.orchestra.run.smithi107.stdout:3591 write 0x8e48e00 thru 0x8e52eb9 (0xa0ba bytes) 2024-09-11T09:29:38.425 INFO:teuthology.orchestra.run.smithi107.stdout:3593 write 0x64f3000 thru 0x64fad13 (0x7d14 bytes) 2024-09-11T09:29:38.425 INFO:teuthology.orchestra.run.smithi107.stdout:3594 punch from 0x72f5600 to 0x72fe000, (0x8a00 bytes) 2024-09-11T09:29:38.432 INFO:teuthology.orchestra.run.smithi107.stdout:3595 trunc from 0x8e53000 to 0xdf7de00 2024-09-11T09:29:38.441 INFO:teuthology.orchestra.run.smithi107.stdout:3596 write 0x491b600 thru 0x49212b5 (0x5cb6 bytes) 2024-09-11T09:29:38.442 INFO:teuthology.orchestra.run.smithi107.stdout:3597 write 0x43b0e00 thru 0x43b3e5b (0x305c bytes) 2024-09-11T09:29:38.442 INFO:teuthology.orchestra.run.smithi107.stdout:3600 write 0x4a45400 thru 0x4a4a92c (0x552d bytes) 2024-09-11T09:29:38.453 INFO:teuthology.orchestra.run.smithi107.stdout:3601 trunc from 0xdf7de00 to 0x4e82c00 2024-09-11T09:29:38.463 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:38.463+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:38.476 INFO:teuthology.orchestra.run.smithi107.stdout:3602 punch from 0x1c41c00 to 0x1c43c00, (0x2000 bytes) 2024-09-11T09:29:38.495 INFO:teuthology.orchestra.run.smithi107.stdout:3603 write 0xd25d200 thru 0xd268dbc (0xbbbd bytes) 2024-09-11T09:29:38.496 INFO:teuthology.orchestra.run.smithi107.stdout:3606 punch from 0x1effa00 to 0x1f0d600, (0xdc00 bytes) 2024-09-11T09:29:38.497 INFO:teuthology.orchestra.run.smithi107.stdout:3607 write 0xa91e200 thru 0xa921ec7 (0x3cc8 bytes) 2024-09-11T09:29:38.497 INFO:teuthology.orchestra.run.smithi107.stdout:3608 write 0x76aa400 thru 0x76af2c6 (0x4ec7 bytes) 2024-09-11T09:29:38.497 INFO:teuthology.orchestra.run.smithi107.stdout:3610 read 0x2bcf200 thru 0x2bd343a (0x423b bytes) 2024-09-11T09:29:38.497 INFO:teuthology.orchestra.run.smithi107.stdout:3611 read 0x2788e00 thru 0x27941e7 (0xb3e8 bytes) 2024-09-11T09:29:38.498 INFO:teuthology.orchestra.run.smithi107.stdout:3612 write 0x86a2200 thru 0x86ad901 (0xb702 bytes) 2024-09-11T09:29:38.498 INFO:teuthology.orchestra.run.smithi107.stdout:3613 punch from 0x86d3400 to 0x86dee00, (0xba00 bytes) 2024-09-11T09:29:38.498 INFO:teuthology.orchestra.run.smithi107.stdout:3614 trunc from 0xd268e00 to 0xca4a000 2024-09-11T09:29:38.516 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:38.516 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:38.517 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:38.517 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:38.521 INFO:teuthology.orchestra.run.smithi107.stdout:3615 trunc from 0xca4a000 to 0x54b6e00 2024-09-11T09:29:38.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:38.534 INFO:teuthology.orchestra.run.smithi107.stdout:3618 punch from 0x3253a00 to 0x325fa00, (0xc000 bytes) 2024-09-11T09:29:38.546 INFO:teuthology.orchestra.run.smithi107.stdout:3622 write 0x6e7da00 thru 0x6e86420 (0x8a21 bytes) 2024-09-11T09:29:38.546 INFO:teuthology.orchestra.run.smithi107.stdout:3623 punch from 0x177ae00 to 0x1782a00, (0x7c00 bytes) 2024-09-11T09:29:38.547 INFO:teuthology.orchestra.run.smithi107.stdout:3627 read 0x26af400 thru 0x26b0f71 (0x1b72 bytes) 2024-09-11T09:29:38.547 INFO:teuthology.orchestra.run.smithi107.stdout:3629 write 0x2f87400 thru 0x2f8d934 (0x6535 bytes) 2024-09-11T09:29:38.547 INFO:teuthology.orchestra.run.smithi107.stdout:3630 punch from 0x3fab600 to 0x3fb9600, (0xe000 bytes) 2024-09-11T09:29:38.547 INFO:teuthology.orchestra.run.smithi107.stdout:3633 write 0x363c600 thru 0x363fcae (0x36af bytes) 2024-09-11T09:29:38.548 INFO:teuthology.orchestra.run.smithi107.stdout:3635 read 0x2376200 thru 0x237f92e (0x972f bytes) 2024-09-11T09:29:38.548 INFO:teuthology.orchestra.run.smithi107.stdout:3636 read 0x2408600 thru 0x240bc70 (0x3671 bytes) 2024-09-11T09:29:38.552 INFO:teuthology.orchestra.run.smithi107.stdout:3638 trunc from 0x6e86600 to 0x9fe6c00 2024-09-11T09:29:38.564 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:38.565+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:38.565 INFO:teuthology.orchestra.run.smithi107.stdout:3640 trunc from 0x9fe6c00 to 0xc777200 2024-09-11T09:29:38.580 INFO:teuthology.orchestra.run.smithi107.stdout:3641 punch from 0x3d65c00 to 0x3d67e00, (0x2200 bytes) 2024-09-11T09:29:38.581 INFO:teuthology.orchestra.run.smithi107.stdout:3643 write 0x3d95e00 thru 0x3d9b71a (0x591b bytes) 2024-09-11T09:29:38.581 INFO:teuthology.orchestra.run.smithi107.stdout:3644 read 0xb23c600 thru 0xb23ce35 (0x836 bytes) 2024-09-11T09:29:38.581 INFO:teuthology.orchestra.run.smithi107.stdout:3646 punch from 0x18d5400 to 0x18d9400, (0x4000 bytes) 2024-09-11T09:29:38.583 INFO:teuthology.orchestra.run.smithi107.stdout:3647 trunc from 0xc777200 to 0xdf81a00 2024-09-11T09:29:38.598 INFO:teuthology.orchestra.run.smithi107.stdout:3648 read 0xc84ca00 thru 0xc8549a3 (0x7fa4 bytes) 2024-09-11T09:29:38.599 INFO:teuthology.orchestra.run.smithi107.stdout:3652 read 0xbced400 thru 0xbcedab8 (0x6b9 bytes) 2024-09-11T09:29:38.599 INFO:teuthology.orchestra.run.smithi107.stdout:3653 read 0x6f9de00 thru 0x6fa23ed (0x45ee bytes) 2024-09-11T09:29:38.607 INFO:teuthology.orchestra.run.smithi107.stdout:3655 trunc from 0xdf81a00 to 0x7288c00 2024-09-11T09:29:38.639 INFO:teuthology.orchestra.run.smithi107.stdout:3657 write 0xdd89a00 thru 0xdd96465 (0xca66 bytes) 2024-09-11T09:29:38.640 INFO:teuthology.orchestra.run.smithi107.stdout:3659 read 0xd50c600 thru 0xd516726 (0xa127 bytes) 2024-09-11T09:29:38.653 INFO:teuthology.orchestra.run.smithi107.stdout:3660 trunc from 0xdd96600 to 0x2447200 2024-09-11T09:29:38.664 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:38.665+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:38.676 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:38.676 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:38.676 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:38.676 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:38.676 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:38.685 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:38.692 INFO:teuthology.orchestra.run.smithi107.stdout:3661 write 0x9d12400 thru 0x9d1c0ec (0x9ced bytes) 2024-09-11T09:29:38.693 INFO:teuthology.orchestra.run.smithi107.stdout:3662 read 0x8929200 thru 0x892dbc2 (0x49c3 bytes) 2024-09-11T09:29:38.693 INFO:teuthology.orchestra.run.smithi107.stdout:3663 write 0x93c00 thru 0x9d471 (0x9872 bytes) 2024-09-11T09:29:38.712 INFO:teuthology.orchestra.run.smithi107.stdout:3664 write 0xbec7400 thru 0xbed3315 (0xbf16 bytes) 2024-09-11T09:29:38.713 INFO:teuthology.orchestra.run.smithi017.stderr:marked in osd.3. 2024-09-11T09:29:38.725 INFO:teuthology.orchestra.run.smithi107.stdout:3665 trunc from 0xbed3400 to 0x1860a00 2024-09-11T09:29:38.730 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-11T09:29:38.771 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:38.772+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:38.796 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:38.796 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.813287434999999, 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:38.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.813265659999999, 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:38.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:38.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.813219616000001, 2024-09-11T09:29:38.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:38.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.813207945999999, 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:38.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.812334537000002, 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.816 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811971062000001, 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:38.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:38.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:38.825 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811919676999999, 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:38.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:38.830 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811883976000001, 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:38.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:38.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:38.835 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811861939, 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:38.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:38.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811826354000001, 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.840 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:38.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811766011, 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:38.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:38.849 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811717327, 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:38.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.853 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811660863, 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:38.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811641307999999, 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:38.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811593545000001, 2024-09-11T09:29:38.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:38.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811479807000001, 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.868 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:38.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.872 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811461062999999, 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.873 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:38.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811422073999999, 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:38.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:38.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:38.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:38.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.811398966999999, 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:38.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.1033224760000007, 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:38.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:38.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:29:38.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:29:38.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:38.893 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:38.893 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:38.894 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:38.894+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:38.894 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:38.995 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:38.996+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:39.003 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:39.003 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:39.003 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:39.003 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:39.012 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:39.096 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:39.097+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:39.118 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:39.118 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:39.118 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:39.118 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:39.119 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:39.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:39.197 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:39.198+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4508.0:2583 3.17 3:ec221060:::rbd_data.1106259a042c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 1362944~13312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 37.604298581999998, 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022291582000000001, 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.276 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4508", 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1955719740", 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2583 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671181+0000", 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.671182+0000", 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1500000000000004e-07 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678960+0000", 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0077783490000000004 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678984+0000", 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3975999999999999e-05 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.678993+0000", 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9819999999999997e-06 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679053+0000", 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9729999999999999e-05 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679166+0000", 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113108 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.679459+0000", 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029270300000000001 2024-09-11T09:29:39.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.688054+0000", 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0085956669999999995 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693292+0000", 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0052377140000000001 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693389+0000", 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7338999999999998e-05 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:01.693473+0000", 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3299000000000003e-05 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.282 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1715 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.399074917999997, 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022344029000000001, 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1715 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.283 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876405+0000", 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.96e-07 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876407+0000", 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.871e-06 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876414+0000", 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4679999999999997e-06 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876419+0000", 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1939999999999997e-06 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898563+0000", 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022144646 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898643+0000", 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9900000000000004e-05 2024-09-11T09:29:39.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898749+0000", 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105554 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1716 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:39.287 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.399054657000001, 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022552065999999999, 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1716 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876425+0000", 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876426+0000", 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.77e-07 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876432+0000", 2024-09-11T09:29:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0079999999999999e-06 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876438+0000", 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4999999999999996e-06 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876443+0000", 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.259e-06 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.290 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898781+0000", 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022337308 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898875+0000", 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4388000000000005e-05 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.898977+0000", 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010222599999999999 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1717 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.399029751999997, 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022753961, 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.292 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1717 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876450+0000", 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876451+0000", 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876452+0000", 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.905e-06 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876459+0000", 2024-09-11T09:29:39.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4980000000000003e-06 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876464+0000", 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1909999999999999e-06 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899008+0000", 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022544281999999999 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899093+0000", 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4505000000000005e-05 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899204+0000", 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111049 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1718 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.399009088, 2024-09-11T09:29:39.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02296196, 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1718 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:39.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876471+0000", 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5999999999999999e-07 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876473+0000", 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9259999999999999e-06 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876480+0000", 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4989999999999999e-06 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876485+0000", 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2159999999999997e-06 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899235+0000", 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022750631 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899321+0000", 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5958999999999995e-05 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899433+0000", 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011126899999999999 2024-09-11T09:29:39.300 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1719 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 262144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398968670999999, 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023149355, 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.301 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1719 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876511+0000", 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.302 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876512+0000", 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.44e-07 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876514+0000", 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876522+0000", 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5000000000000002e-06 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.303 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876527+0000", 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.587e-06 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899467+0000", 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.022939975000000001 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899552+0000", 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5131999999999997e-05 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899660+0000", 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108137 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1720 3.12 3:4b812eac:::rbd_data.1106cda8c42d.0000000000000002:a [read 284672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398945081000001, 2024-09-11T09:29:39.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023387654000000001, 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1720 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876535+0000", 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.63e-07 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876537+0000", 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1040000000000001e-06 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876544+0000", 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7800000000000003e-06 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876549+0000", 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2549999999999997e-06 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899706+0000", 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023156708000000002 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899801+0000", 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4826000000000003e-05 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899922+0000", 2024-09-11T09:29:39.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121518 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1760 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 12288~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398924188999999, 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023618561999999999, 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1760 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876556+0000", 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876558+0000", 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1670000000000002e-06 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876566+0000", 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3810000000000001e-06 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876571+0000", 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3029999999999999e-06 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.899963+0000", 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023392445000000001 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900061+0000", 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7495000000000001e-05 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900174+0000", 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000113391 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1761 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 34816~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398902389, 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023824222999999999, 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1761 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876577+0000", 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876578+0000", 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876580+0000", 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9539999999999998e-06 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876586+0000", 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3149999999999997e-06 2024-09-11T09:29:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876591+0000", 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900220+0000", 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023628546 2024-09-11T09:29:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900317+0000", 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7168000000000002e-05 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900402+0000", 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4701000000000001e-05 2024-09-11T09:29:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1762 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 57344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398882133999997, 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024033523000000001, 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1762 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876598+0000", 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876600+0000", 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.052e-06 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876607+0000", 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876612+0000", 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9259999999999999e-06 2024-09-11T09:29:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900448+0000", 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023836570000000001 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900534+0000", 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5314000000000003e-05 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900631+0000", 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7676000000000005e-05 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1763 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 79872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398861707999998, 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024166084000000001, 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.325 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1763 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876618+0000", 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876619+0000", 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6400000000000003e-07 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876621+0000", 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0949999999999998e-06 2024-09-11T09:29:39.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876629+0000", 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8380000000000008e-06 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876635+0000", 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3199999999999999e-06 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900675+0000", 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024040611 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900724+0000", 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8097999999999999e-05 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900784+0000", 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0658000000000001e-05 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.329 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1764 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 102400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398838470999998, 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024356692999999999, 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.330 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1764 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876641+0000", 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876642+0000", 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876644+0000", 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.041e-06 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876651+0000", 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6309999999999997e-06 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876656+0000", 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1399999999999999e-06 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900814+0000", 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024158129 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900906+0000", 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2618e-05 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.900998+0000", 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1624000000000002e-05 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1765 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 126976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:39.334 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398817741000002, 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024574166000000001, 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1765 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876662+0000", 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.39e-07 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876664+0000", 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9520000000000001e-06 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876671+0000", 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3670000000000002e-06 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876676+0000", 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901029+0000", 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024353631000000001 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901109+0000", 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9951999999999996e-05 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901236+0000", 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012697000000000001 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398797952000002, 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:39.339 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:39.343 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398777719999998, 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.344 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:39.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398751593999997, 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:39.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:39.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.353 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398729308999997, 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:39.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:39.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398705868999997, 2024-09-11T09:29:39.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:39.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:39.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:39.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:39.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398686022, 2024-09-11T09:29:39.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:39.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:39.368 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 33.398653645000003, 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.369 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:39.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.373 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:39.375 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:39.375 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:39.375+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:39.475 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:39.476+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:1303 3.4 3.d53d3ee4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:38.727423+0000", 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.79775638599999998, 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.79778200399999999, 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for peered", 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1303 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727423+0000", 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727423+0000", 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727425+0000", 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.601e-06 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727433+0000", 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3990000000000004e-06 2024-09-11T09:29:39.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727448+0000", 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5421e-05 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727457+0000", 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1300000000000001e-06 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727514+0000", 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7336999999999997e-05 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:38.727523+0000", 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4630000000000007e-06 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 1 2024-09-11T09:29:39.528 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:39.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:39.576 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:39.577+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:39.639 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:39.640 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:39.640 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:39.640 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:39.640 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:39.648 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:39.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:39.691 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:39.692+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:39.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:29:39.762 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.851331104, 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:39.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:39.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.851100532, 2024-09-11T09:29:39.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:39.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:39.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.851003328000001, 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.772 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:39.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:39.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.850768563999999, 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:39.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.781 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.850572547999999, 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.850304111, 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:39.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.850202361000001, 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:39.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.849808704000001, 2024-09-11T09:29:39.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:39.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:39.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.849682459, 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.800 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.804 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.849473076999999, 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:39.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:39.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:39.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.849362127999999, 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:39.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:39.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.849106190000001, 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.814 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:39.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:39.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.818 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.848953660999999, 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:39.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:39.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:39.823 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.848761296999999, 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:39.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:39.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.848430878999999, 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.828 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:39.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.848309326999999, 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.833 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:39.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.848154549, 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:39.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:39.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.848100629000001, 2024-09-11T09:29:39.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:39.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:39.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:39.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.847900060000001, 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.847 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:39.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:39.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 28.847544521, 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:39.852 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:39.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:39.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:39.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:39.857 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:39.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:39.859 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:39.859 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:39.859 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:39.860 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:39.860 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:39.860 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:39.858+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:39.874 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:39.874 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:39.893 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:39.893 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:39.912 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:39.912 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:39.965 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:39.966+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:39.983 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:39.983 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:39.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:40.060 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:40.060+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:40.139 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.140 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.148 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:40.192 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:40.193+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:40.255 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.255 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.262 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:40.262+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:40.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:40.363 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:40.364+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:40.378 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.378 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:40.463 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:40.464+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:40.492 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.492 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:40.564 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:40.565+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:40.602 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.602 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.610 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:40.665 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:40.665+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:40.713 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.713 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.720 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:29:40.722 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:40.785 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:40.786+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:40.789 INFO:teuthology.orchestra.run.smithi107.stdout:3667 read 0x4b6c00 thru 0x4c0ee0 (0xa2e1 bytes) 2024-09-11T09:29:40.791 INFO:teuthology.orchestra.run.smithi107.stdout:3668 punch from 0x14c6600 to 0x14d0a00, (0xa400 bytes) 2024-09-11T09:29:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:3672 read 0xaec400 thru 0xaed8e5 (0x14e6 bytes) 2024-09-11T09:29:40.792 INFO:teuthology.orchestra.run.smithi107.stdout:3678 read 0xb5c800 thru 0xb61c4f (0x5450 bytes) 2024-09-11T09:29:40.819 INFO:teuthology.orchestra.run.smithi107.stdout:3680 write 0xa11c400 thru 0xa127d7e (0xb97f bytes) 2024-09-11T09:29:40.820 INFO:teuthology.orchestra.run.smithi107.stdout:3683 read 0x208ca00 thru 0x20953ff (0x8a00 bytes) 2024-09-11T09:29:40.824 INFO:teuthology.orchestra.run.smithi107.stdout:3684 punch from 0x51800 to 0x57400, (0x5c00 bytes) 2024-09-11T09:29:40.825 INFO:teuthology.orchestra.run.smithi107.stdout:3685 read 0x913f600 thru 0x9144079 (0x4a7a bytes) 2024-09-11T09:29:40.825 INFO:teuthology.orchestra.run.smithi107.stdout:3688 read 0x8586600 thru 0x858e8b0 (0x82b1 bytes) 2024-09-11T09:29:40.825 INFO:teuthology.orchestra.run.smithi107.stdout:3691 write 0x652a200 thru 0x653039e (0x619f bytes) 2024-09-11T09:29:40.825 INFO:teuthology.orchestra.run.smithi107.stdout:3692 punch from 0x1c01e00 to 0x1c07a00, (0x5c00 bytes) 2024-09-11T09:29:40.842 INFO:teuthology.orchestra.run.smithi107.stdout:3694 write 0xa173200 thru 0xa17f36d (0xc16e bytes) 2024-09-11T09:29:40.848 INFO:teuthology.orchestra.run.smithi107.stdout:3695 read 0x60b0000 thru 0x60be05f (0xe060 bytes) 2024-09-11T09:29:40.848 INFO:teuthology.orchestra.run.smithi107.stdout:3696 punch from 0x2104200 to 0x210b200, (0x7000 bytes) 2024-09-11T09:29:40.848 INFO:teuthology.orchestra.run.smithi107.stdout:3697 read 0x2a8bc00 thru 0x2a96b8d (0xaf8e bytes) 2024-09-11T09:29:40.849 INFO:teuthology.orchestra.run.smithi107.stdout:3698 punch from 0x3cba00 to 0x3d0200, (0x4800 bytes) 2024-09-11T09:29:40.852 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.853 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:40.863 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:40.864 INFO:teuthology.orchestra.run.smithi107.stdout:3699 trunc from 0xa17f400 to 0xd64a00 2024-09-11T09:29:40.866 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:40.866+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:40.886 INFO:teuthology.orchestra.run.smithi107.stdout:3701 read 0x6a1800 thru 0x6ae560 (0xcd61 bytes) 2024-09-11T09:29:40.887 INFO:teuthology.orchestra.run.smithi107.stdout:3702 punch from 0x91f000 to 0x92b200, (0xc200 bytes) 2024-09-11T09:29:40.887 INFO:teuthology.orchestra.run.smithi107.stdout:3707 punch from 0x735e00 to 0x742c00, (0xce00 bytes) 2024-09-11T09:29:40.888 INFO:teuthology.orchestra.run.smithi107.stdout:3708 punch from 0x67c400 to 0x689400, (0xd000 bytes) 2024-09-11T09:29:40.888 INFO:teuthology.orchestra.run.smithi107.stdout:3709 read 0xa9b600 thru 0xa9bf64 (0x965 bytes) 2024-09-11T09:29:40.888 INFO:teuthology.orchestra.run.smithi107.stdout:3710 read 0x60b200 thru 0x614785 (0x9586 bytes) 2024-09-11T09:29:40.889 INFO:teuthology.orchestra.run.smithi107.stdout:3714 read 0x313400 thru 0x314c82 (0x1883 bytes) 2024-09-11T09:29:40.918 INFO:teuthology.orchestra.run.smithi107.stdout:3715 write 0xe48d600 thru 0xe496186 (0x8b87 bytes) 2024-09-11T09:29:40.933 INFO:teuthology.orchestra.run.smithi107.stdout:3716 write 0xe806c00 thru 0xe80e20b (0x760c bytes) 2024-09-11T09:29:40.934 INFO:teuthology.orchestra.run.smithi107.stdout:3721 read 0x552e200 thru 0x5536bc7 (0x89c8 bytes) 2024-09-11T09:29:40.934 INFO:teuthology.orchestra.run.smithi107.stdout:3723 read 0x8239600 thru 0x8242eb0 (0x98b1 bytes) 2024-09-11T09:29:40.938 INFO:teuthology.orchestra.run.smithi107.stdout:3724 read 0x1845c00 thru 0x184dcb2 (0x80b3 bytes) 2024-09-11T09:29:40.938 INFO:teuthology.orchestra.run.smithi107.stdout:3726 read 0x23ed400 thru 0x23f82b0 (0xaeb1 bytes) 2024-09-11T09:29:40.938 INFO:teuthology.orchestra.run.smithi107.stdout:3727 trunc from 0xe80e400 to 0xd6bdc00 2024-09-11T09:29:40.954 INFO:teuthology.orchestra.run.smithi107.stdout:3728 write 0x6a000 thru 0x784c2 (0xe4c3 bytes) 2024-09-11T09:29:40.962 INFO:teuthology.orchestra.run.smithi107.stdout:3729 read 0x59b4800 thru 0x59bc54c (0x7d4d bytes) 2024-09-11T09:29:40.962 INFO:teuthology.orchestra.run.smithi107.stdout:3734 read 0xc67c400 thru 0xc67e9b7 (0x25b8 bytes) 2024-09-11T09:29:40.965 INFO:teuthology.orchestra.run.smithi107.stdout:3735 trunc from 0xd6bdc00 to 0xaa37000 2024-09-11T09:29:40.966 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:40.967+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:40.990 INFO:teuthology.orchestra.run.smithi107.stdout:3736 trunc from 0xaa37000 to 0x185200 2024-09-11T09:29:40.990 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:40.991 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:41.012 INFO:teuthology.orchestra.run.smithi107.stdout:3739 read 0x9ae00 thru 0xa1e41 (0x7042 bytes) 2024-09-11T09:29:41.013 INFO:teuthology.orchestra.run.smithi107.stdout:3740 punch from 0x4f600 to 0x59e00, (0xa800 bytes) 2024-09-11T09:29:41.024 INFO:teuthology.orchestra.run.smithi107.stdout:3743 trunc from 0x185200 to 0x8d7e600 2024-09-11T09:29:41.046 INFO:teuthology.orchestra.run.smithi107.stdout:3744 write 0xc5d5c00 thru 0xc5df970 (0x9d71 bytes) 2024-09-11T09:29:41.047 INFO:teuthology.orchestra.run.smithi107.stdout:3745 write 0x17fe00 thru 0x187b49 (0x7d4a bytes) 2024-09-11T09:29:41.048 INFO:teuthology.orchestra.run.smithi107.stdout:3746 write 0x362c800 thru 0x3632e34 (0x6635 bytes) 2024-09-11T09:29:41.048 INFO:teuthology.orchestra.run.smithi107.stdout:3747 read 0x3d40600 thru 0x3d4efb1 (0xe9b2 bytes) 2024-09-11T09:29:41.048 INFO:teuthology.orchestra.run.smithi107.stdout:3748 read 0x1295600 thru 0x12a334b (0xdd4c bytes) 2024-09-11T09:29:41.061 INFO:teuthology.orchestra.run.smithi107.stdout:3749 trunc from 0xc5dfa00 to 0x14a4800 2024-09-11T09:29:41.068 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:41.068+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:41.085 INFO:teuthology.orchestra.run.smithi107.stdout:3750 trunc from 0x14a4800 to 0x1528800 2024-09-11T09:29:41.095 INFO:teuthology.orchestra.run.smithi107.stdout:3751 trunc from 0x1528800 to 0x1a92200 2024-09-11T09:29:41.121 INFO:teuthology.orchestra.run.smithi107.stdout:3752 trunc from 0x1a92200 to 0xb022a00 2024-09-11T09:29:41.136 INFO:teuthology.orchestra.run.smithi107.stdout:3753 read 0x56cd400 thru 0x56d4b4f (0x7750 bytes) 2024-09-11T09:29:41.137 INFO:teuthology.orchestra.run.smithi107.stdout:3757 read 0x79c7400 thru 0x79cec40 (0x7841 bytes) 2024-09-11T09:29:41.137 INFO:teuthology.orchestra.run.smithi107.stdout:3759 read 0x973b000 thru 0x974930c (0xe30d bytes) 2024-09-11T09:29:41.148 INFO:teuthology.orchestra.run.smithi107.stdout:3760 write 0xd6b2c00 thru 0xd6b47bf (0x1bc0 bytes) 2024-09-11T09:29:41.148 INFO:teuthology.orchestra.run.smithi107.stdout:3762 punch from 0x6ccb600 to 0x6cd5c00, (0xa600 bytes) 2024-09-11T09:29:41.154 INFO:teuthology.orchestra.run.smithi107.stdout:3765 trunc from 0xd6b4800 to 0x8d04c00 2024-09-11T09:29:41.164 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.164 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.168 INFO:teuthology.orchestra.run.smithi107.stdout:3766 write 0xe3b400 thru 0xe41e7c (0x6a7d bytes) 2024-09-11T09:29:41.169 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:41.169+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:3767 write 0x2293600 thru 0x22a19f6 (0xe3f7 bytes) 2024-09-11T09:29:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:3768 read 0x21dac00 thru 0x21e5d92 (0xb193 bytes) 2024-09-11T09:29:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:3770 read 0x8b46800 thru 0x8b492e1 (0x2ae2 bytes) 2024-09-11T09:29:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:3772 write 0x66f5400 thru 0x66fe4c0 (0x90c1 bytes) 2024-09-11T09:29:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:3775 read 0x1f61000 thru 0x1f6b808 (0xa809 bytes) 2024-09-11T09:29:41.170 INFO:teuthology.orchestra.run.smithi107.stdout:3776 clone 23 order 18 su 65536 sc 4 2024-09-11T09:29:41.174 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:41.270 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:41.271+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:41.286 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.287 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.296 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:41.371 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:41.372+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:41.407 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.407 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:41.472 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:41.472+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:41.539 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.540 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:41.592 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:41.593+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:41.658 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.658 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:41.709 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:41.709+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:41.774 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:41.775+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:41.819 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.819 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:41.875 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:41.876+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:41.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:41.976 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:41.977+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:41.989 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:41.989 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:41.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:42.056 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:42.057 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:42.075 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:42.076 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:42.076 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:42.077+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:42.092 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:42.092 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:42.109 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:42.109 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:42.126 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:42.126 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:42.143 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:42.143 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:42.162 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:42.162 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:42.162 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:42.163 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:42.163 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:42.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:42.213 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:42.214+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:42.279 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:42.280+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.411724436, 2024-09-11T09:29:42.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.411493864000001, 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.328 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:42.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:42.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.411396660000001, 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.333 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:42.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.337 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.411161895999999, 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:42.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.410965879999999, 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:42.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.346 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.410697443, 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.347 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:42.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.351 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.410595693000001, 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:42.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.410202036000001, 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.356 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:42.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.410075791000001, 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:42.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.409866408999999, 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:42.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:42.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.40975546, 2024-09-11T09:29:42.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.409499522000001, 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:42.379 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.409346993, 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:42.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:42.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.409154629, 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:42.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:42.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.408824210999999, 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.389 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:42.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3988 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 450560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:42.393 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.408702658999999, 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814902999999998, 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3988 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:42.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913970+0000", 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913971+0000", 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913974+0000", 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6560000000000001e-06 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913979+0000", 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1379999999999998e-06 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913982+0000", 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2780000000000002e-06 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952691+0000", 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038709186 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952731+0000", 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9787999999999999e-05 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952785+0000", 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4171e-05 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.397 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.408547881000001, 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:42.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:42.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.408493961000001, 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.406 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.408293392000001, 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:42.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.407937853, 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:42.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.415 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:42.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:42.417 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:42.417+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:42.517 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:42.518+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:42.566 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:42.567 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:42.567 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:42.567 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:42.575 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:42.618 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:42.619+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:42.682 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:42.682 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:42.682 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:42.682 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:42.682 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:42.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:42.734 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:42.735+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:42.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:29:42.820 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:42.821+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 66.382791439000002, 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:42.842 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:42.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.121271751000002, 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:42.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:42.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.855 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.120025783999999, 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:42.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:42.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:42.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:42.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:42.860 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:42.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:42.921 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:42.921+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:42.967 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:42.967 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:42.975 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:43.021 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:43.021+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:43.117 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:43.117 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:43.117 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:43.118 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:43.118 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:43.122 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:43.123+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:43.126 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:43.223 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:43.224+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:43.229 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 72.791991646, 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:43.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:43.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5081 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267716452000002, 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043672824999999998, 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5081 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961275+0000", 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961281+0000", 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2840000000000003e-06 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961285+0000", 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3560000000000001e-06 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961287+0000", 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5409999999999999e-06 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004781+0000", 2024-09-11T09:29:43.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043493472999999998 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004856+0000", 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5741000000000005e-05 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004947+0000", 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1222000000000002e-05 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5083 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 907264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267700877999999, 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043785219, 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5083 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.242 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961290+0000", 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961291+0000", 2024-09-11T09:29:43.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5699999999999999e-07 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961292+0000", 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961295+0000", 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0149999999999999e-06 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961297+0000", 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.004977+0000", 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043679791000000003 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005021+0000", 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4295000000000002e-05 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005075+0000", 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4299999999999998e-05 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5088 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 905216~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267691331000002, 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043963918999999997, 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5088 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961300+0000", 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4200000000000002e-07 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961301+0000", 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.13e-06 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961305+0000", 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4529999999999998e-06 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961309+0000", 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9709999999999998e-06 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005096+0000", 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043787848999999997 2024-09-11T09:29:43.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005175+0000", 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8487999999999998e-05 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005264+0000", 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8786e-05 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5097 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1605632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267654483999998, 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044049360000000003, 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5097 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961337+0000", 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0999999999999999e-07 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961339+0000", 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1440000000000001e-06 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961345+0000", 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7470000000000002e-06 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961350+0000", 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2209999999999996e-06 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005291+0000", 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043940405000000002 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005332+0000", 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1542999999999998e-05 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005386+0000", 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3789999999999998e-05 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5100 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1617920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267634633, 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044229475999999997, 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5100 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961356+0000", 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961357+0000", 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961359+0000", 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.052e-06 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961365+0000", 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8030000000000002e-06 2024-09-11T09:29:43.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961369+0000", 2024-09-11T09:29:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6689999999999997e-06 2024-09-11T09:29:43.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005422+0000", 2024-09-11T09:29:43.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044052701 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005499+0000", 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6664999999999999e-05 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005586+0000", 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7230000000000003e-05 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5105 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1630208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267616052000001, 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044370891000000003, 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5105 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961375+0000", 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.58e-07 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961377+0000", 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9120000000000001e-06 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961383+0000", 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6890000000000001e-06 2024-09-11T09:29:43.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961388+0000", 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.792e-06 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005629+0000", 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044240743999999999 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005687+0000", 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7951000000000001e-05 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005746+0000", 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9444999999999998e-05 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5108 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1607680~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267597694999999, 2024-09-11T09:29:43.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581704, 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5108 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961393+0000", 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961394+0000", 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961396+0000", 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961401+0000", 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8119999999999996e-06 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961406+0000", 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4920000000000004e-06 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005789+0000", 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044383498 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005882+0000", 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2541999999999996e-05 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.005975+0000", 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.3110999999999997e-05 2024-09-11T09:29:43.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5111 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1619968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267579650999998, 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044694581999999997, 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5111 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961411+0000", 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961412+0000", 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961414+0000", 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.996e-06 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961420+0000", 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8000000000000004e-06 2024-09-11T09:29:43.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961425+0000", 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5149999999999997e-06 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006007+0000", 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044581786999999998 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006051+0000", 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3998999999999997e-05 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006106+0000", 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5145999999999997e-05 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5116 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1632256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:43.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267560246999999, 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044886862999999999, 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5116 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:43.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961431+0000", 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.39e-07 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961433+0000", 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961439+0000", 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8860000000000001e-06 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961444+0000", 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5809999999999998e-06 2024-09-11T09:29:43.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006144+0000", 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044699782 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006221+0000", 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7270999999999999e-05 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006318+0000", 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6914000000000001e-05 2024-09-11T09:29:43.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5121 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1611776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267541825999999, 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045003543, 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5121 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961449+0000", 2024-09-11T09:29:43.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961450+0000", 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0800000000000001e-07 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961452+0000", 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0030000000000001e-06 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961459+0000", 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1860000000000003e-06 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961464+0000", 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006348+0000", 2024-09-11T09:29:43.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044884563000000002 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006396+0000", 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7817e-05 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006453+0000", 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6762000000000003e-05 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5124 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1609728~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267521357, 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045118734000000001, 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5124 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961470+0000", 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961472+0000", 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8929999999999999e-06 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961478+0000", 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.592e-06 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961482+0000", 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5220000000000001e-06 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006470+0000", 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.044987680000000002 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006529+0000", 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9116000000000001e-05 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006589+0000", 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9463000000000003e-05 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5129 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1624064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267502987999997, 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045244390000000002, 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5129 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961488+0000", 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3500000000000002e-07 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961490+0000", 2024-09-11T09:29:43.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9850000000000001e-06 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961496+0000", 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4920000000000001e-06 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961500+0000", 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3819999999999997e-06 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006606+0000", 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045106159 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006669+0000", 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2188999999999997e-05 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006733+0000", 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3848000000000003e-05 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5134 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1636352~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267468016999999, 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045344917999999998, 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5134 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961523+0000", 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961524+0000", 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7399999999999998e-07 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961531+0000", 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.968e-06 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961541+0000", 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0059e-05 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961546+0000", 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0679999999999996e-06 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006750+0000", 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045204167000000003 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006808+0000", 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8229000000000001e-05 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006868+0000", 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9953e-05 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5135 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1622016~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267438316000003, 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045554071000000002, 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5135 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.298 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961553+0000", 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7100000000000002e-07 2024-09-11T09:29:43.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961555+0000", 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1679999999999998e-06 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961561+0000", 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0310000000000004e-06 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961566+0000", 2024-09-11T09:29:43.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4170000000000002e-06 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.006908+0000", 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045342423999999999 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007004+0000", 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5307000000000001e-05 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007107+0000", 2024-09-11T09:29:43.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000103253 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5138 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1634304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267418546999998, 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045679006000000001, 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.302 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5138 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961573+0000", 2024-09-11T09:29:43.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961805+0000", 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023282700000000001 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961816+0000", 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0648999999999999e-05 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961828+0000", 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2318e-05 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961835+0000", 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007139+0000", 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045303738000000003 2024-09-11T09:29:43.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007188+0000", 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9014999999999997e-05 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007252+0000", 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3921e-05 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5143 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1613824~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.267145163000002, 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045634479999999998, 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5143 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.307 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961846+0000", 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961994+0000", 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000147949 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.961997+0000", 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5580000000000001e-06 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962007+0000", 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6579999999999997e-06 2024-09-11T09:29:43.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962014+0000", 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007290+0000", 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045276416 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007380+0000", 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9686999999999997e-05 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007480+0000", 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100762 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5148 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1626112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.266967272999999, 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045674845999999998, 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5148 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962024+0000", 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962412+0000", 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00038829000000000001 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962417+0000", 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4009999999999999e-06 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962427+0000", 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0477e-05 2024-09-11T09:29:43.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962434+0000", 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7379999999999997e-06 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007514+0000", 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045080264000000002 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007600+0000", 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.5970000000000005e-05 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007699+0000", 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8706e-05 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5153 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1615872~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.266545862000001, 2024-09-11T09:29:43.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045448919999999997, 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5153 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962445+0000", 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962446+0000", 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962448+0000", 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.531e-06 2024-09-11T09:29:43.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962455+0000", 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5440000000000004e-06 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962458+0000", 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6339999999999999e-06 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007712+0000", 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045253466999999999 2024-09-11T09:29:43.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007784+0000", 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1939000000000004e-05 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007894+0000", 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000110301 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:5156 3.a 3:501082bf:::rbd_data.11069296af76.0000000000000006:b [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.266526804000002, 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045635835999999999, 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 5156 2024-09-11T09:29:43.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962464+0000", 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962886+0000", 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00042155799999999999 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962890+0000", 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-06 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962899+0000", 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.0440000000000001e-06 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.962906+0000", 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1169999999999998e-06 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007917+0000", 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.045011412000000001 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.007994+0000", 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6650999999999995e-05 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.008100+0000", 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000106164 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.325 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:43.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:43.327 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:43.327+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:43.427 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:43.428+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:43.469 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:43.470 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:43.478 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:43.527 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:43.528+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:43.581 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:43.581 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:43.582 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:43.582 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:43.582 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:43.590 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:43.628 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:43.629+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:43.687 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:29:43.731 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:43.732 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:43.732 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T09:29:43.732 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.468799 2024-09-11T09:29:43.732 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.46879871401206374 2024-09-11T09:29:43.740 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:43.740 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373149169000001, 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:43.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373111784000002, 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:43.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373092466000003, 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:43.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:43.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373081220000003, 2024-09-11T09:29:43.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:43.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373039315, 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:43.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373021911000002, 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.373004016000003, 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:43.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372983836000003, 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372962852000001, 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:43.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:43.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:43.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372935755, 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:43.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:43.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.788 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372916468, 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:43.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372712108000002, 2024-09-11T09:29:43.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372178351999999, 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:43.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372129459, 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:43.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:43.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372081371, 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.807 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:43.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.372043333000001, 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:43.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:43.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.371819490999997, 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.817 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:43.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.371642389000002, 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:43.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.371496833999998, 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:43.826 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:43.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 40.371375448000002, 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:43.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:43.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:43.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:43.835 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:43.836 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone22 2024-09-11T09:29:43.836 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:43.751+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:43.837 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:43.837+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:43.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:43.937 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:43.938+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:43.944 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:43.944 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:43.945 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:43.945 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:43.953 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:44.038 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:44.038+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:44.100 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:44.100 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:44.100 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:44.100 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:44.100 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:44.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:44.138 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:44.139+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:44.152 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-11T09:29:44.158 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:44.216 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.232811648999999, 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:44.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.232789873999998, 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:44.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.232743829999997, 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:44.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.232732159999998, 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231858750999997, 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:44.236 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:44.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231495275999997, 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:44.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231443890999998, 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231408190000003, 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231386153000003, 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231350568000003, 2024-09-11T09:29:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231290225000002, 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231241541000003, 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231185076999999, 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231165521999998, 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231117759, 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.231004020999997, 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.230985277000002, 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.230946287999998, 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 33.230923181000001, 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.52284669, 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:29:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:29:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:44.313 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:44.314 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:44.314 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:44.315 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:44.315 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:44.315 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:44.314+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:44.316 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:44.327 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:44.327 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:44.347 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:44.367 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:44.386 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:44.387 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:44.415 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:44.416+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:44.438 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:44.438 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:44.448 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:44.516 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:44.516+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:44.602 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:44.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:44.655 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:44.655+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:44.718 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:44.718 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:44.718 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:44.719+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:44.726 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:44.738 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.1 primary-affinity to 0.468799 (8307232) 2024-09-11T09:29:44.768 INFO:teuthology.orchestra.run.smithi107.stdout:3779 read 0x32db000 thru 0x32ea247 (0xf248 bytes) 2024-09-11T09:29:44.770 INFO:teuthology.orchestra.run.smithi107.stdout:3780 trunc from 0x8d04c00 to 0x7994600 2024-09-11T09:29:44.819 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:44.820+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:44.847 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:44.847 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:44.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:44.920 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:44.921+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:44.963 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:44.963 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:44.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:45.056 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:45.057+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:45.119 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.119 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.122 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:45.122+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:45.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:45.222 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:45.223+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:45.275 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.275 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:45.323 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:45.323+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:45.424 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:45.425+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:45.431 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.432 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:45.526 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:45.527+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:45.547 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.547 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:45.627 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:45.628+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:45.661 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.661 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:45.709 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:29:45.753 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:45.754+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:45.755 INFO:teuthology.orchestra.run.smithi107.stdout:3781 read 0x342de00 thru 0x3435667 (0x7868 bytes) 2024-09-11T09:29:45.762 INFO:teuthology.orchestra.run.smithi107.stdout:3782 read 0x4001a00 thru 0x400c11f (0xa720 bytes) 2024-09-11T09:29:45.762 INFO:teuthology.orchestra.run.smithi107.stdout:3784 punch from 0x13f3400 to 0x13f3a00, (0x600 bytes) 2024-09-11T09:29:45.763 INFO:teuthology.orchestra.run.smithi107.stdout:3786 write 0x6e21e00 thru 0x6e2a0a4 (0x82a5 bytes) 2024-09-11T09:29:45.763 INFO:teuthology.orchestra.run.smithi107.stdout:3787 read 0x3f7bc00 thru 0x3f8a315 (0xe716 bytes) 2024-09-11T09:29:45.764 INFO:teuthology.orchestra.run.smithi107.stdout:3788 read 0xebbe00 thru 0xec1560 (0x5761 bytes) 2024-09-11T09:29:45.764 INFO:teuthology.orchestra.run.smithi107.stdout:3789 punch from 0x1d0cc00 to 0x1d0f200, (0x2600 bytes) 2024-09-11T09:29:45.770 INFO:teuthology.orchestra.run.smithi107.stdout:3790 trunc from 0x7994600 to 0x4ce8a00 2024-09-11T09:29:45.782 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.782 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:45.829 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:45.829+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:45.829 INFO:teuthology.orchestra.run.smithi107.stdout:3792 write 0xd45e200 thru 0xd461e2b (0x3c2c bytes) 2024-09-11T09:29:45.830 INFO:teuthology.orchestra.run.smithi107.stdout:3793 read 0x293a400 thru 0x2945d1c (0xb91d bytes) 2024-09-11T09:29:45.830 INFO:teuthology.orchestra.run.smithi107.stdout:3794 read 0x4220600 thru 0x422a0ac (0x9aad bytes) 2024-09-11T09:29:45.840 INFO:teuthology.orchestra.run.smithi107.stdout:3796 trunc from 0xd462000 to 0x5367200 2024-09-11T09:29:45.881 INFO:teuthology.orchestra.run.smithi107.stdout:3797 write 0xdfdd800 thru 0xdfe4544 (0x6d45 bytes) 2024-09-11T09:29:45.889 INFO:teuthology.orchestra.run.smithi107.stdout:3798 write 0x3f12800 thru 0x3f21a0a (0xf20b bytes) 2024-09-11T09:29:45.889 INFO:teuthology.orchestra.run.smithi107.stdout:3799 write 0x4c66200 thru 0x4c749bb (0xe7bc bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3800 read 0xdb89000 thru 0xdb90c04 (0x7c05 bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3801 read 0xa761e00 thru 0xa76a083 (0x8284 bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3802 write 0x1270000 thru 0x127d744 (0xd745 bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3803 write 0x40a4200 thru 0x40b38c8 (0xf6c9 bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3805 read 0x2d37200 thru 0x2d39500 (0x2301 bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3807 read 0x8eaf000 thru 0x8eba26e (0xb26f bytes) 2024-09-11T09:29:45.890 INFO:teuthology.orchestra.run.smithi107.stdout:3808 clone 24 order 25 su 2097152 sc 14 2024-09-11T09:29:45.906 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:45.906 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:45.916 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:45.929 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:45.930+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:46.017 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:46.018 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:46.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:46.029 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:46.030+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:46.128 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:46.129 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:46.131 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:46.131+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:46.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:46.231 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:46.233+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:46.281 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:46.281 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:46.290 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:46.332 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:46.333+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:46.393 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:46.393 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:46.400 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:46.402 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:46.446 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:46.447+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:46.504 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:46.504 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:46.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:46.534 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:46.535+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:46.537 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:46.537 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:46.555 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:46.555 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:46.571 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:46.572 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:46.588 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:46.589 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:46.605 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:46.605 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:46.623 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:46.623 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:46.659 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4839 3.1c 3:3c27467a:::rbd_data.1106cda8c42d.000000000000000c:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.676608862999998, 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033794151000000001, 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4839 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983099+0000", 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983102+0000", 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2299999999999998e-06 2024-09-11T09:29:46.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983108+0000", 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2049999999999999e-06 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983113+0000", 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6080000000000003e-06 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016789+0000", 2024-09-11T09:29:46.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033675991000000002 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016831+0000", 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1996000000000002e-05 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016893+0000", 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1597e-05 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.676587087999998, 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:46.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.669 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.676541043999997, 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:46.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.676529373999998, 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:46.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:46.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.678 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4871 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 290816~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675655964999997, 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033801652000000001, 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4871 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984052+0000", 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3200000000000005e-07 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984055+0000", 2024-09-11T09:29:46.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.322e-06 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984062+0000", 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3889999999999999e-06 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984067+0000", 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4360000000000001e-06 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017757+0000", 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033689209999999997 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017800+0000", 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3204000000000002e-05 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017853+0000", 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3559e-05 2024-09-11T09:29:46.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4930 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 487424~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675292489999997, 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033768982000000003, 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4930 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984415+0000", 2024-09-11T09:29:46.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984416+0000", 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984418+0000", 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1229999999999998e-06 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984424+0000", 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5549999999999999e-06 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984429+0000", 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.938e-06 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018102+0000", 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033673017 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018146+0000", 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3441e-05 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018184+0000", 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8566000000000003e-05 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675241104999998, 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.689 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:46.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:46.693 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675205404000003, 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:46.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:46.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675183367000002, 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:46.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675147782000003, 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:46.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675087439000002, 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:46.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:46.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.675038755000003, 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:46.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674982290999999, 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:46.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:46.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:46.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674962735999998, 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:46.721 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:46.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:46.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674914973, 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674801234999997, 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:46.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:46.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674782491000002, 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.740 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674743501999998, 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:46.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:46.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:46.745 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 35.674720395000001, 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:46.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:29:46.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 16.966643904000001, 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:29:46.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:46.755 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:46.756 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:46.676+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:46.757 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:29:46.757 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:46.757+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:46.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:46.857 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:46.858+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:46.866 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:46.866 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:46.866 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:46.866 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:46.875 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:46.958 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:46.959+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:47.020 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone23 from 0xdfe4600 (overlap 0x4ce8a00) to 0x0 2024-09-11T09:29:47.024 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:47.024 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:47.024 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:47.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:47.065 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:47.063+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:47.160 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:47.161+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:47.177 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300580683, 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.178 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:47.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:47.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300560451000003, 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.183 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:47.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:47.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300534325000001, 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.188 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:47.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300512040000001, 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.193 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:47.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.197 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300488600000001, 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.198 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:47.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:47.202 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300468752999997, 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.300436376, 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:548 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.150942404, 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026786889000000001, 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 548 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:29:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026321+0000", 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8599999999999996e-07 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026326+0000", 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2669999999999997e-06 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026342+0000", 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5695000000000001e-05 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026350+0000", 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026418+0000", 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7268999999999996e-05 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026621+0000", 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000203172 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053107+0000", 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026486129000000001 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:558 3.1f 3:fa19977a:::rbd_data.1106b643ff7f.000000000000001a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.15079983799999999, 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026276384, 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 558 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1200000000000002e-07 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026468+0000", 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2729999999999996e-06 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026508+0000", 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0095000000000002e-05 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026516+0000", 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6549999999999993e-06 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026689+0000", 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017269599999999999 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026791+0000", 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010203 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.052739+0000", 2024-09-11T09:29:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025947923000000001 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:565 3.1f 3:fa508b4a:::rbd_data.1106b643ff7f.000000000000001e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.15072255600000001, 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026702457999999998, 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 565 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026541+0000", 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9999999999999997e-07 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026546+0000", 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6829999999999996e-06 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026554+0000", 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6869999999999994e-06 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026559+0000", 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027294+0000", 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000734361 2024-09-11T09:29:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027468+0000", 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017458400000000001 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053242+0000", 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025774047000000001 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:595 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.15031240900000001, 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026285078, 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 595 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026951+0000", 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3799999999999997e-07 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026954+0000", 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0599999999999999e-06 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026961+0000", 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3189999999999997e-06 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026967+0000", 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5890000000000002e-06 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027011+0000", 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4682999999999999e-05 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027107+0000", 2024-09-11T09:29:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5419000000000002e-05 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053235+0000", 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026128470000000001 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:629 3.1f 3:fb0b56ac:::rbd_data.1106b643ff7f.0000000000000048:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.14987914999999999, 2024-09-11T09:29:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02599951, 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 629 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:29:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027384+0000", 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.23e-07 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027391+0000", 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0899999999999999e-06 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027408+0000", 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.717e-05 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027416+0000", 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1570000000000003e-06 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027791+0000", 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00037519499999999999 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027902+0000", 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111215 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053383+0000", 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025480860000000001 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:638 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.149774556, 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025900833000000002, 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 638 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027489+0000", 2024-09-11T09:29:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0299999999999998e-07 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027492+0000", 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6270000000000002e-06 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027501+0000", 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8440000000000004e-06 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027507+0000", 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0689999999999998e-06 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028168+0000", 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066121099999999998 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028276+0000", 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107435 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053389+0000", 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025112944000000002 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:660 3.12 3:4f6d0c71:::rbd_data.1106b643ff7f.000000000000005c:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.149117369, 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025370475, 2024-09-11T09:29:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 660 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:29:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028146+0000", 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2900000000000004e-07 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028149+0000", 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9000000000000002e-06 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028156+0000", 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2200000000000003e-06 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028161+0000", 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6910000000000002e-06 2024-09-11T09:29:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028580+0000", 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000418625 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028685+0000", 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010467300000000001 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053516+0000", 2024-09-11T09:29:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024830837000000001 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:973 3.1f 3:ff421b5d:::rbd_data.1106b643ff7f.000000000000008d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.054727+0000", 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.12253523299999999, 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024593184000000001, 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 973 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054727+0000", 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054727+0000", 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054728+0000", 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6000000000000004e-07 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054730+0000", 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.43e-06 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054737+0000", 2024-09-11T09:29:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4479999999999999e-06 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054742+0000", 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1560000000000003e-06 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055595+0000", 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00085359900000000002 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055680+0000", 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4019999999999999e-05 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079320+0000", 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023640971 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1000 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.122280467, 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037281880000000003, 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1000 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054983+0000", 2024-09-11T09:29:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054987+0000", 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1670000000000001e-06 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054996+0000", 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7490000000000004e-06 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055002+0000", 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4799999999999998e-06 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055397+0000", 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00039521700000000001 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055474+0000", 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6872000000000005e-05 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055665+0000", 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000191013 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056748+0000", 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010830779999999999 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092163+0000", 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035415042000000001 2024-09-11T09:29:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092225+0000", 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1555000000000003e-05 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092264+0000", 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9020999999999998e-05 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1010 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.122123516, 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037320416000000002, 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1010 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2199999999999999e-07 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055142+0000", 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5620000000000002e-06 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055148+0000", 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2400000000000004e-06 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055153+0000", 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.977e-06 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056168+0000", 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010149040000000001 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056237+0000", 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8606e-05 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056413+0000", 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017639199999999999 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057731+0000", 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001317616 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092370+0000", 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034638872000000001 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092419+0000", 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8989999999999997e-05 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092459+0000", 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0834999999999998e-05 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1011 3.1f 3:fb0b56ac:::rbd_data.1106b643ff7f.0000000000000048:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.055160+0000", 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.122102424, 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024720552, 2024-09-11T09:29:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1011 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055160+0000", 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055160+0000", 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055161+0000", 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4700000000000002e-07 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055163+0000", 2024-09-11T09:29:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7630000000000001e-06 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055171+0000", 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3050000000000004e-06 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055176+0000", 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.933e-06 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057813+0000", 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0026371580000000001 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057920+0000", 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107329 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.058116+0000", 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00019628500000000001 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.058691+0000", 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000574381 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079757+0000", 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.021066685000000002 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079827+0000", 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0098999999999994e-05 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079881+0000", 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3167000000000001e-05 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1018 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:29:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.120275058, 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035657162999999999, 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1018 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056988+0000", 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.95e-07 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056994+0000", 2024-09-11T09:29:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4500000000000003e-06 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057005+0000", 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1522000000000001e-05 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057013+0000", 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6599999999999995e-06 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057058+0000", 2024-09-11T09:29:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057153+0000", 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5377000000000005e-05 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057338+0000", 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018502199999999999 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057925+0000", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00058726299999999998 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092552+0000", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034627003000000003 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092598+0000", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5989000000000003e-05 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092645+0000", 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6431999999999999e-05 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1086 3.12 3:4b95dab8:::rbd_data.1106b643ff7f.0000000000000085:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.116621553, 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.032140305000000001, 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1086 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:29:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060642+0000", 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4300000000000002e-07 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060650+0000", 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1969999999999996e-06 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060660+0000", 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0471e-05 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060667+0000", 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4829999999999996e-06 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060798+0000", 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013138599999999999 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060884+0000", 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5348999999999999e-05 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061065+0000", 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018090700000000001 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061845+0000", 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00078057400000000002 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092700+0000", 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.030854940000000001 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092749+0000", 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9101999999999998e-05 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092781+0000", 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2053e-05 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.278 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:47.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:47.279 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:47.279+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:47.379 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:47.380+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:47.432 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:47.432 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:47.432 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:47.432 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:47.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:47.467 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-11T09:29:47.482 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:47.483+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:47.580 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:47.581+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:47.596 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:47.596 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:47.596 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:47.596 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:47.596 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:47.605 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:47.681 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:47.682+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:47.760 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:47.760 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:47.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:47.760 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:47.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.84865052, 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:47.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.848419948, 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:47.765 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:47.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.848322744000001, 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.770 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:47.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.848087980000003, 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.847891963999999, 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:47.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:47.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:47.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.783 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.847623527000003, 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:47.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:47.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.847521776999997, 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:47.788 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.847128120000001, 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.847001874999997, 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.846792493000002, 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.846681543999999, 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.808 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:47.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.846425605999997, 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:47.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:47.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.817 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.846273076999999, 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:47.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:47.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.846080712999999, 2024-09-11T09:29:47.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:47.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:47.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.845750295000002, 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.827 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:47.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.845473964999996, 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:47.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:47.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.836 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.845420044999997, 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.845219475999997, 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.842 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:47.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:47.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.844863937, 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:47.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:47.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:47.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.850 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.0655984040000002, 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:47.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:29:47.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:29:47.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:29:47.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:47.857 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:47.859 INFO:teuthology.orchestra.run.smithi107.stdout:3809 write 0x349ec00 thru 0x34a0e82 (0x2283 bytes) 2024-09-11T09:29:47.859 INFO:teuthology.orchestra.run.smithi107.stdout:3811 write 0x3804400 thru 0x3812e1e (0xea1f bytes) 2024-09-11T09:29:47.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:47.860 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:47.860+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:47.862 INFO:teuthology.orchestra.run.smithi107.stdout:3812 trunc from 0xdfe4600 to 0x750a800 2024-09-11T09:29:47.959 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:47.960+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:47.988 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:47.988 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:47.988 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:47.988 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:47.997 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:47.998 INFO:teuthology.orchestra.run.smithi107.stdout:3815 write 0xb8f0e00 thru 0xb8ffb38 (0xed39 bytes) 2024-09-11T09:29:47.998 INFO:teuthology.orchestra.run.smithi107.stdout:3816 write 0xb170a00 thru 0xb1753e9 (0x49ea bytes) 2024-09-11T09:29:47.998 INFO:teuthology.orchestra.run.smithi107.stdout:3819 read 0x2911c00 thru 0x291d3e6 (0xb7e7 bytes) 2024-09-11T09:29:47.998 INFO:teuthology.orchestra.run.smithi107.stdout:3824 read 0x2408600 thru 0x240f02a (0x6a2b bytes) 2024-09-11T09:29:47.999 INFO:teuthology.orchestra.run.smithi107.stdout:3825 read 0x48a9e00 thru 0x48b10be (0x72bf bytes) 2024-09-11T09:29:47.999 INFO:teuthology.orchestra.run.smithi107.stdout:3829 write 0x3b71c00 thru 0x3b740d1 (0x24d2 bytes) 2024-09-11T09:29:48.010 INFO:teuthology.orchestra.run.smithi107.stdout:3831 trunc from 0xb8ffc00 to 0x3143e00 2024-09-11T09:29:48.042 INFO:teuthology.orchestra.run.smithi107.stdout:3833 write 0x2cc8600 thru 0x2cc9dc1 (0x17c2 bytes) 2024-09-11T09:29:48.060 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:48.061+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:48.062 INFO:teuthology.orchestra.run.smithi107.stdout:3835 write 0x5e84200 thru 0x5e9267e (0xe47f bytes) 2024-09-11T09:29:48.064 INFO:teuthology.orchestra.run.smithi107.stdout:3836 trunc from 0x5e92800 to 0x71e8a00 2024-09-11T09:29:48.074 INFO:teuthology.orchestra.run.smithi107.stdout:3837 punch from 0x46cbe00 to 0x46cf000, (0x3200 bytes) 2024-09-11T09:29:48.083 INFO:teuthology.orchestra.run.smithi107.stdout:3838 trunc from 0x71e8a00 to 0xccb400 2024-09-11T09:29:48.152 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:48.153 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:48.154 INFO:teuthology.orchestra.run.smithi107.stdout:3840 write 0x4658800 thru 0x4667117 (0xe918 bytes) 2024-09-11T09:29:48.161 INFO:teuthology.orchestra.run.smithi107.stdout:3845 read 0x1afaa00 thru 0x1b01a6a (0x706b bytes) 2024-09-11T09:29:48.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:48.162 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:48.162+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:48.189 INFO:teuthology.orchestra.run.smithi107.stdout:3846 write 0xd6b3000 thru 0xd6c2f23 (0xff24 bytes) 2024-09-11T09:29:48.189 INFO:teuthology.orchestra.run.smithi107.stdout:3847 write 0x1451c00 thru 0x1461422 (0xf823 bytes) 2024-09-11T09:29:48.189 INFO:teuthology.orchestra.run.smithi107.stdout:3848 read 0x3030e00 thru 0x3039e52 (0x9053 bytes) 2024-09-11T09:29:48.202 INFO:teuthology.orchestra.run.smithi107.stdout:3850 trunc from 0xd6c3000 to 0x26ba600 2024-09-11T09:29:48.241 INFO:teuthology.orchestra.run.smithi107.stdout:3851 read 0x2211800 thru 0x2218a9c (0x729d bytes) 2024-09-11T09:29:48.241 INFO:teuthology.orchestra.run.smithi107.stdout:3853 punch from 0x2673600 to 0x2679c00, (0x6600 bytes) 2024-09-11T09:29:48.241 INFO:teuthology.orchestra.run.smithi107.stdout:3854 read 0x1ef4600 thru 0x1f03372 (0xed73 bytes) 2024-09-11T09:29:48.248 INFO:teuthology.orchestra.run.smithi107.stdout:3855 trunc from 0x26ba600 to 0x890f400 2024-09-11T09:29:48.262 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:48.263+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:48.272 INFO:teuthology.orchestra.run.smithi107.stdout:3856 trunc from 0x890f400 to 0x1252c00 2024-09-11T09:29:48.302 INFO:teuthology.orchestra.run.smithi107.stdout:3859 read 0xb6c600 thru 0xb6fc97 (0x3698 bytes) 2024-09-11T09:29:48.315 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:48.315 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4208.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.856553379999994, 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00065323200000000001, 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.316 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460306+0000", 2024-09-11T09:29:48.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8799999999999998e-07 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460316+0000", 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6299999999999993e-06 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460325+0000", 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3409999999999992e-06 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460333+0000", 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460383+0000", 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0055000000000001e-05 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460408+0000", 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4331000000000001e-05 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460925+0000", 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0005174 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:36.460959+0000", 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3816000000000003e-05 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:48.320 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.595033692000001, 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:48.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:48.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:48.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop_reply(client.4358.0:13 3.1e e24/21 ondisk, result = 0)", 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.593787724999999, 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000125375, 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:48.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723071+0000", 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723072+0000", 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9599999999999998e-07 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723076+0000", 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6490000000000001e-06 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723084+0000", 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7439999999999996e-06 2024-09-11T09:29:48.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723089+0000", 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723121+0000", 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2240000000000003e-05 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723127+0000", 2024-09-11T09:29:48.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1420000000000001e-06 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723197+0000", 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0102000000000001e-05 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:48.331 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:48.332 INFO:teuthology.orchestra.run.smithi107.stdout:3861 trunc from 0x1252c00 to 0xe63c000 2024-09-11T09:29:48.332 INFO:teuthology.orchestra.run.smithi107.stdout:3862 read 0xa687c00 thru 0xa691179 (0x957a bytes) 2024-09-11T09:29:48.333 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:48.341 INFO:teuthology.orchestra.run.smithi107.stdout:3863 trunc from 0xe63c000 to 0x33aee00 2024-09-11T09:29:48.363 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:48.363+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:48.367 INFO:teuthology.orchestra.run.smithi107.stdout:3867 punch from 0x1307200 to 0x130d200, (0x6000 bytes) 2024-09-11T09:29:48.381 INFO:teuthology.orchestra.run.smithi107.stdout:3868 trunc from 0x33aee00 to 0x9ee5000 2024-09-11T09:29:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:3870 read 0x3819200 thru 0x3826729 (0xd52a bytes) 2024-09-11T09:29:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:3871 read 0x37f3600 thru 0x37f76f0 (0x40f1 bytes) 2024-09-11T09:29:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:3872 punch from 0x3cb8200 to 0x3cbc600, (0x4400 bytes) 2024-09-11T09:29:48.397 INFO:teuthology.orchestra.run.smithi107.stdout:3873 read 0x912a200 thru 0x912beb8 (0x1cb9 bytes) 2024-09-11T09:29:48.406 INFO:teuthology.orchestra.run.smithi107.stdout:3874 trunc from 0x9ee5000 to 0x2db2200 2024-09-11T09:29:48.430 INFO:teuthology.orchestra.run.smithi107.stdout:3876 read 0x21ada00 thru 0x21b1dae (0x43af bytes) 2024-09-11T09:29:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:3877 read 0x2837600 thru 0x283c81f (0x5220 bytes) 2024-09-11T09:29:48.431 INFO:teuthology.orchestra.run.smithi107.stdout:3882 read 0x245f200 thru 0x2462888 (0x3689 bytes) 2024-09-11T09:29:48.444 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:48.444 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:48.444 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:48.444 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:48.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:48.458 INFO:teuthology.orchestra.run.smithi107.stdout:3883 write 0x6b0c000 thru 0x6b0c740 (0x741 bytes) 2024-09-11T09:29:48.464 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:48.465+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:48.480 INFO:teuthology.orchestra.run.smithi107.stdout:3884 write 0xee18e00 thru 0xee216b1 (0x88b2 bytes) 2024-09-11T09:29:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:3885 trunc from 0xee21800 to 0x1cc2e00 2024-09-11T09:29:48.543 INFO:teuthology.orchestra.run.smithi107.stdout:3887 trunc from 0x1cc2e00 to 0xcb6ac00 2024-09-11T09:29:48.553 INFO:teuthology.orchestra.run.smithi107.stdout:3888 punch from 0x62d600 to 0x633200, (0x5c00 bytes) 2024-09-11T09:29:48.555 INFO:teuthology.orchestra.run.smithi107.stdout:3890 read 0x3c7ea00 thru 0x3c8e5b0 (0xfbb1 bytes) 2024-09-11T09:29:48.556 INFO:teuthology.orchestra.run.smithi107.stdout:3892 read 0x110d600 thru 0x11187cf (0xb1d0 bytes) 2024-09-11T09:29:48.556 INFO:teuthology.orchestra.run.smithi107.stdout:3893 read 0x86e6800 thru 0x86f0be7 (0xa3e8 bytes) 2024-09-11T09:29:48.564 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:48.564 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:48.564 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:48.564 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:48.564 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:48.565 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:48.565+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:48.567 INFO:teuthology.orchestra.run.smithi107.stdout:3896 trunc from 0xcb6ac00 to 0x3139400 2024-09-11T09:29:48.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:48.619 INFO:teuthology.orchestra.run.smithi107.stdout:3897 write 0x34e9200 thru 0x34f8d78 (0xfb79 bytes) 2024-09-11T09:29:48.619 INFO:teuthology.orchestra.run.smithi107.stdout:3901 read 0x31efa00 thru 0x31f66cf (0x6cd0 bytes) 2024-09-11T09:29:48.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:48.644 INFO:teuthology.orchestra.run.smithi107.stdout:3902 write 0xb82c400 thru 0xb8391c6 (0xcdc7 bytes) 2024-09-11T09:29:48.645 INFO:teuthology.orchestra.run.smithi107.stdout:3908 punch from 0x984e200 to 0x9853200, (0x5000 bytes) 2024-09-11T09:29:48.647 INFO:teuthology.orchestra.run.smithi107.stdout:3914 read 0x1c3b800 thru 0x1c3f1f4 (0x39f5 bytes) 2024-09-11T09:29:48.656 INFO:teuthology.orchestra.run.smithi107.stdout:3915 trunc from 0xb839200 to 0x3d6ea00 2024-09-11T09:29:48.666 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:48.667+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:48.677 INFO:teuthology.orchestra.run.smithi107.stdout:3916 punch from 0x3685200 to 0x3694e00, (0xfc00 bytes) 2024-09-11T09:29:48.682 INFO:teuthology.orchestra.run.smithi107.stdout:3919 read 0x2d56a00 thru 0x2d60fa1 (0xa5a2 bytes) 2024-09-11T09:29:48.683 INFO:teuthology.orchestra.run.smithi107.stdout:3920 punch from 0x1659c00 to 0x165de00, (0x4200 bytes) 2024-09-11T09:29:48.683 INFO:teuthology.orchestra.run.smithi107.stdout:3923 read 0xcae000 thru 0xcb3d5b (0x5d5c bytes) 2024-09-11T09:29:48.683 INFO:teuthology.orchestra.run.smithi107.stdout:3924 read 0x891800 thru 0x898702 (0x6f03 bytes) 2024-09-11T09:29:48.690 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:48.690 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:48.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.252811394000005, 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:48.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:48.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:48.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.636905667, 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:29:48.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:29:48.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:29:48.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6367079659999999, 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:29:48.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.710 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6364495400000001, 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:29:48.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:29:48.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6364196559999999, 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:29:48.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:48.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6358865040000001, 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:48.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:29:48.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.635009055, 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:29:48.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6349700889999998, 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:29:48.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.634926613, 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:29:48.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.634801996, 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:29:48.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.634236526, 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:29:48.744 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:29:48.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:29:48.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.63420589, 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.749 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:29:48.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6320694580000001, 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:29:48.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:29:48.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.63054897, 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:29:48.758 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:29:48.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.762 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.626475141, 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.763 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:29:48.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:48.767 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.626361347, 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:48.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:29:48.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.626276665, 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:29:48.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6262306739999999, 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:29:48.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:29:48.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:48.784 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.625859376, 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:29:48.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.6257174270000001, 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:48.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:48.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:29:48.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:48.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:48.794 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:48.794 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:29:48.795 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:48.796 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:48.797 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:48.797 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:48.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:48.798 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:48.796+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:48.798 INFO:teuthology.orchestra.run.smithi107.stdout:3925 trunc from 0x3d6ea00 to 0xeae4000 2024-09-11T09:29:48.798 INFO:teuthology.orchestra.run.smithi107.stdout:3928 read 0x87bbe00 thru 0x87c20a5 (0x62a6 bytes) 2024-09-11T09:29:48.798 INFO:teuthology.orchestra.run.smithi107.stdout:3930 write 0x31a8200 thru 0x31ad382 (0x5183 bytes) 2024-09-11T09:29:48.798 INFO:teuthology.orchestra.run.smithi107.stdout:3931 trunc from 0xeae4000 to 0x2bee000 2024-09-11T09:29:48.798 INFO:teuthology.orchestra.run.smithi107.stdout:3932 write 0x14b8c00 thru 0x14bbfdf (0x33e0 bytes) 2024-09-11T09:29:48.798 INFO:teuthology.orchestra.run.smithi107.stdout:3935 punch from 0xce8000 to 0xcf1a00, (0x9a00 bytes) 2024-09-11T09:29:48.799 INFO:teuthology.orchestra.run.smithi107.stdout:3936 clone 25 order 23 su 2097152 sc 13 2024-09-11T09:29:48.799 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:48.799 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:48.814 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:48.814 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:48.831 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:48.831 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:48.848 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:48.848 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:48.896 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:48.897+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:48.941 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:48.941 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:48.949 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:48.997 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:48.997+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:49.094 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.094 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.097 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:49.098+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:49.103 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:49.198 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:49.199+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:49.206 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.206 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.215 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:49.299 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:49.300+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:49.358 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.358 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.367 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:49.407 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:49.408+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:49.501 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:49.502+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:49.508 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.508 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.517 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:49.602 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:49.603+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:49.661 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.661 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:49.711 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:49.712+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:49.751 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:29:49.754 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:49.754 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:49.754 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-11T09:29:49.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-11T09:29:49.789 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.789 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:49.805 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:49.805+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:49.837 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone24 2024-09-11T09:29:49.905 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:49.906+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:49.948 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:49.948 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:49.958 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:50.006 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:50.007+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:50.067 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-11T09:29:50.068+0000 7fef02752640 -1 mon.a@0(leader).osd e71 definitely_dead 0 2024-09-11T09:29:50.068 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.068 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:50.120 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:50.121+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:50.129 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-11T09:29:50.185 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.185 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.188 INFO:teuthology.orchestra.run.smithi107.stdout:3937 read 0x1c42e00 thru 0x1c52086 (0xf287 bytes) 2024-09-11T09:29:50.189 INFO:teuthology.orchestra.run.smithi107.stdout:3938 punch from 0x712400 to 0x71ee00, (0xca00 bytes) 2024-09-11T09:29:50.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:50.207 INFO:teuthology.orchestra.run.smithi107.stdout:3939 trunc from 0x2bee000 to 0x1c2ca00 2024-09-11T09:29:50.208 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:50.208+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:50.235 INFO:teuthology.orchestra.run.smithi107.stdout:3941 read 0x185c000 thru 0x1867163 (0xb164 bytes) 2024-09-11T09:29:50.242 INFO:teuthology.orchestra.run.smithi107.stdout:3943 read 0xcf1a00 thru 0xcf700d (0x560e bytes) 2024-09-11T09:29:50.242 INFO:teuthology.orchestra.run.smithi107.stdout:3946 write 0x9b7600 thru 0x9be356 (0x6d57 bytes) 2024-09-11T09:29:50.251 INFO:teuthology.orchestra.run.smithi107.stdout:3948 trunc from 0x1c2ca00 to 0x89ec000 2024-09-11T09:29:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:3951 read 0xec6600 thru 0xed0668 (0xa069 bytes) 2024-09-11T09:29:50.263 INFO:teuthology.orchestra.run.smithi107.stdout:3952 read 0x178b000 thru 0x1793682 (0x8683 bytes) 2024-09-11T09:29:50.263 INFO:teuthology.orchestra.run.smithi107.stdout:3953 read 0x246b000 thru 0x2474f5c (0x9f5d bytes) 2024-09-11T09:29:50.271 INFO:teuthology.orchestra.run.smithi107.stdout:3955 trunc from 0x89ec000 to 0x1fc5000 2024-09-11T09:29:50.308 INFO:teuthology.orchestra.run.smithi107.stdout:3957 trunc from 0x1fc5000 to 0xa50c800 2024-09-11T09:29:50.309 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:50.309+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:50.317 INFO:teuthology.orchestra.run.smithi107.stdout:3958 punch from 0x7102a00 to 0x710bc00, (0x9200 bytes) 2024-09-11T09:29:50.337 INFO:teuthology.orchestra.run.smithi107.stdout:3961 write 0xcad1200 thru 0xcae0428 (0xf229 bytes) 2024-09-11T09:29:50.338 INFO:teuthology.orchestra.run.smithi107.stdout:3962 read 0x750dc00 thru 0x750ef5c (0x135d bytes) 2024-09-11T09:29:50.346 INFO:teuthology.orchestra.run.smithi107.stdout:3963 trunc from 0xcae0600 to 0x64a7600 2024-09-11T09:29:50.347 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.347 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.357 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:50.376 INFO:teuthology.orchestra.run.smithi107.stdout:3964 trunc from 0x64a7600 to 0xb883200 2024-09-11T09:29:50.385 INFO:teuthology.orchestra.run.smithi107.stdout:3965 punch from 0xb768400 to 0xb774000, (0xbc00 bytes) 2024-09-11T09:29:50.386 INFO:teuthology.orchestra.run.smithi107.stdout:3967 punch from 0x58cea00 to 0x58db600, (0xcc00 bytes) 2024-09-11T09:29:50.387 INFO:teuthology.orchestra.run.smithi107.stdout:3969 write 0xa6cf800 thru 0xa6d35e5 (0x3de6 bytes) 2024-09-11T09:29:50.387 INFO:teuthology.orchestra.run.smithi107.stdout:3970 write 0x4d39000 thru 0x4d455d0 (0xc5d1 bytes) 2024-09-11T09:29:50.388 INFO:teuthology.orchestra.run.smithi107.stdout:3971 read 0x3d1e600 thru 0x3d2688b (0x828c bytes) 2024-09-11T09:29:50.388 INFO:teuthology.orchestra.run.smithi107.stdout:3972 read 0x9f12000 thru 0x9f17489 (0x548a bytes) 2024-09-11T09:29:50.388 INFO:teuthology.orchestra.run.smithi107.stdout:3973 read 0xb0cd000 thru 0xb0d9b35 (0xcb36 bytes) 2024-09-11T09:29:50.388 INFO:teuthology.orchestra.run.smithi107.stdout:3974 read 0x7a1e000 thru 0x7a2bb13 (0xdb14 bytes) 2024-09-11T09:29:50.388 INFO:teuthology.orchestra.run.smithi107.stdout:3975 read 0x3bfda00 thru 0x3bff700 (0x1d01 bytes) 2024-09-11T09:29:50.389 INFO:teuthology.orchestra.run.smithi107.stdout:3976 read 0x1490200 thru 0x1493b6f (0x3970 bytes) 2024-09-11T09:29:50.389 INFO:teuthology.orchestra.run.smithi107.stdout:3978 punch from 0x182ec00 to 0x1837e00, (0x9200 bytes) 2024-09-11T09:29:50.390 INFO:teuthology.orchestra.run.smithi107.stdout:3979 trunc from 0xb883200 to 0xaa49200 2024-09-11T09:29:50.407 INFO:teuthology.orchestra.run.smithi107.stdout:3980 trunc from 0xaa49200 to 0xecc3a00 2024-09-11T09:29:50.410 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:50.410+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:50.416 INFO:teuthology.orchestra.run.smithi107.stdout:3985 read 0x1f5ca00 thru 0x1f640d5 (0x76d6 bytes) 2024-09-11T09:29:50.417 INFO:teuthology.orchestra.run.smithi107.stdout:3986 read 0x1714600 thru 0x171a2ea (0x5ceb bytes) 2024-09-11T09:29:50.417 INFO:teuthology.orchestra.run.smithi107.stdout:3987 read 0x9c65e00 thru 0x9c69a29 (0x3c2a bytes) 2024-09-11T09:29:50.418 INFO:teuthology.orchestra.run.smithi107.stdout:3988 write 0xbde3400 thru 0xbdee65e (0xb25f bytes) 2024-09-11T09:29:50.418 INFO:teuthology.orchestra.run.smithi107.stdout:3992 write 0x56a9000 thru 0x56afed7 (0x6ed8 bytes) 2024-09-11T09:29:50.419 INFO:teuthology.orchestra.run.smithi107.stdout:3993 read 0x5dade00 thru 0x5dbb607 (0xd808 bytes) 2024-09-11T09:29:50.419 INFO:teuthology.orchestra.run.smithi107.stdout:3994 read 0x8ac3a00 thru 0x8ac5cfb (0x22fc bytes) 2024-09-11T09:29:50.435 INFO:teuthology.orchestra.run.smithi107.stdout:3996 trunc from 0xecc3a00 to 0xc74e00 2024-09-11T09:29:50.474 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.474 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:50.488 INFO:teuthology.orchestra.run.smithi107.stdout:3997 write 0x85dec00 thru 0x85ebdfc (0xd1fd bytes) 2024-09-11T09:29:50.509 INFO:teuthology.orchestra.run.smithi107.stdout:3998 write 0xa3bc000 thru 0xa3cb9d7 (0xf9d8 bytes) 2024-09-11T09:29:50.509 INFO:teuthology.orchestra.run.smithi107.stdout:3999 clone 26 order 21 su 16384 sc 4 2024-09-11T09:29:50.510 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:50.511+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:50.593 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.593 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:50.611 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:50.612+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:50.709 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.727 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:50.729 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:50.729+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:50.762 INFO:teuthology.orchestra.run.smithi017.stderr:marked in osd.2. 2024-09-11T09:29:50.777 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-11T09:29:50.829 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:50.830+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:50.860 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:50.869 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.870 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:50.930 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:50.930+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:50.982 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:50.983 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:50.983 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:50.983 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:50.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:50.999 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:50.999 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:51.015 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:51.031 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:51.031+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:51.033 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:51.033 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:51.049 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:51.049 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:51.067 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:51.068 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:51.097 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:51.097 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:51.097 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:51.105 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:51.148 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:51.149+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:51.209 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 80.772037202999996, 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1561314759999997, 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:29:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:29:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1559337750000003, 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:29:51.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.155675349, 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:29:51.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1556454650000001, 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:29:51.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:29:51.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1551123130000001, 2024-09-11T09:29:51.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:51.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:29:51.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1542348640000002, 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:29:51.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1541958980000002, 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:29:51.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1541524220000001, 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.254 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:51.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:29:51.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1540278050000001, 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:29:51.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:29:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1534623350000004, 2024-09-11T09:29:51.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:29:51.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1534316990000004, 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:29:51.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1512952670000001, 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:29:51.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:29:51.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1497747790000004, 2024-09-11T09:29:51.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:29:51.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:29:51.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1457009500000002, 2024-09-11T09:29:51.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:51.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:29:51.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1455871560000004, 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:29:51.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:29:51.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1455024739999997, 2024-09-11T09:29:51.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:51.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:29:51.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:29:51.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1454564830000002, 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:29:51.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.304 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1450851850000001, 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:51.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.1449432359999996, 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:51.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:29:51.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.314 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.314 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:51.315 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:51.316 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:51.315+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:51.416 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:51.417+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:51.458 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:51.459 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:51.459 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:51.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:51.517 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:51.517+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:51.570 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:51.570 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:51.570 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:51.578 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:51.623 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:51.623+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:51.682 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315158908000001, 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315121523000002, 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:51.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315102205000002, 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:51.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315090959000003, 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:51.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:51.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315049053999999, 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:51.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315031650000002, 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:51.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:51.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:51.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.315013755000003, 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:51.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314993575000003, 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:51.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314972591, 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:51.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314945494, 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:51.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314926206999999, 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:51.730 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314721847000001, 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:51.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314188090999998, 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:51.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314139197999999, 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:51.744 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:51.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.31409111, 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:51.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.314053072, 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:51.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:51.758 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.313829230000003, 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:51.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:51.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.313652128000001, 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.763 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:51.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:51.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.313506572999998, 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:51.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 48.313385187000002, 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:51.772 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:51.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:51.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:51.777 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:51.777 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:51.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:29:51.779 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:51.779 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:51.778+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:51.879 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:51.880+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:51.924 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:51.924 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:51.925 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:51.925 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:51.933 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:51.980 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:51.981+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:52.076 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:52.077 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:52.077 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:52.081 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:52.081+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:52.085 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:52.182 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:52.183+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:52.229 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.246341248999997, 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:29:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.246295205000003, 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:29:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.246283535000003, 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244995265999997, 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:29:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244959565000002, 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:29:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:29:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244937528000001, 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:29:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:29:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244901943000002, 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:29:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244841600000001, 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:29:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244792916000002, 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:29:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244736451999998, 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:29:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:29:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244716897000004, 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244669133999999, 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:29:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244555396000003, 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:29:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:29:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244536652000001, 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244497662999997, 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:29:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:29:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.244474556, 2024-09-11T09:29:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:29:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.536398065, 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:29:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:29:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1739987349999996, 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:29:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:29:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.168747658, 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:29:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 5.1684872879999997, 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:52.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:29:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:52.328 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:52.330 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:52.330+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:52.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:52.430 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:52.431+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5323.0:832 3.18 3.1e206e18 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.67497780900000004, 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.67499906700000001, 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for peered", 2024-09-11T09:29:52.440 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5323", 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2665032934", 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 832 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765940+0000", 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765952+0000", 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1708999999999999e-05 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765970+0000", 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8459999999999999e-05 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765984+0000", 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4056999999999999e-05 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766066+0000", 2024-09-11T09:29:52.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1976000000000003e-05 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766075+0000", 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2940000000000006e-06 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768269+0000", 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021941510000000001 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768283+0000", 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3966e-05 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5323.0:833 3.18 3.1e206e18 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.67489860099999999, 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.67496903799999997, 2024-09-11T09:29:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for peered", 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5323", 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2665032934", 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 833 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:29:52.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766018+0000", 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6199999999999996e-07 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766023+0000", 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8239999999999999e-06 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766034+0000", 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1231e-05 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766042+0000", 2024-09-11T09:29:52.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6720000000000004e-06 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766151+0000", 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109078 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766156+0000", 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6729999999999997e-06 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768984+0000", 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0028281130000000002 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768994+0000", 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0012e-05 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:16052 3.18 3.1e206e18 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.67486103200000003, 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.67497277200000005, 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "waiting for peered", 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 16052 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.451 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766056+0000", 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7460000000000001e-06 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766063+0000", 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7499999999999997e-06 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766078+0000", 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4866e-05 2024-09-11T09:29:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766087+0000", 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4180000000000003e-06 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766271+0000", 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018336900000000001 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:29:52.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766278+0000", 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9009999999999997e-06 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.769020+0000", 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0027424229999999999 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.769028+0000", 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7190000000000001e-06 2024-09-11T09:29:52.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.456 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.456 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:29:52.456 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 3 2024-09-11T09:29:52.456 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:52.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:52.531 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:52.531+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:52.604 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:52.604 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:52.604 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:52.604 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:52.604 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:52.613 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:52.632 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:52.633+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1766 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 149504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845803502000003, 2024-09-11T09:29:52.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024714113999999999, 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1766 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.724 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876682+0000", 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876684+0000", 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9570000000000001e-06 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876691+0000", 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3770000000000001e-06 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876696+0000", 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901269+0000", 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024572931999999999 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901334+0000", 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5291999999999998e-05 2024-09-11T09:29:52.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901396+0000", 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1993000000000001e-05 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845783269999998, 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:29:52.728 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845757143999997, 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.733 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.737 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845734858999997, 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:29:52.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845711418999997, 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:29:52.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:29:52.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845691572, 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.747 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:29:52.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.751 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.845659195000003, 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:29:52.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:29:52.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:548 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6961652230000004, 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026786889000000001, 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 548 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026321+0000", 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8599999999999996e-07 2024-09-11T09:29:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026326+0000", 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2669999999999997e-06 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026342+0000", 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5695000000000001e-05 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026350+0000", 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026418+0000", 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7268999999999996e-05 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026621+0000", 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000203172 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053107+0000", 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026486129000000001 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:558 3.1f 3:fa19977a:::rbd_data.1106b643ff7f.000000000000001a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6960226570000003, 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026276384, 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 558 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1200000000000002e-07 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026468+0000", 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2729999999999996e-06 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026508+0000", 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0095000000000002e-05 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026516+0000", 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6549999999999993e-06 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026689+0000", 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017269599999999999 2024-09-11T09:29:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026791+0000", 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010203 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.052739+0000", 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025947923000000001 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:565 3.1f 3:fa508b4a:::rbd_data.1106b643ff7f.000000000000001e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.695945375, 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026702457999999998, 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 565 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026541+0000", 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9999999999999997e-07 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026546+0000", 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6829999999999996e-06 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026554+0000", 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6869999999999994e-06 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026559+0000", 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:29:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027294+0000", 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000734361 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027468+0000", 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017458400000000001 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053242+0000", 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025774047000000001 2024-09-11T09:29:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:595 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6955352279999998, 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026285078, 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.770 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 595 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026951+0000", 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3799999999999997e-07 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026954+0000", 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0599999999999999e-06 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026961+0000", 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3189999999999997e-06 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026967+0000", 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5890000000000002e-06 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027011+0000", 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4682999999999999e-05 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027107+0000", 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5419000000000002e-05 2024-09-11T09:29:52.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053235+0000", 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026128470000000001 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:629 3.1f 3:fb0b56ac:::rbd_data.1106b643ff7f.0000000000000048:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6951019689999995, 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02599951, 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 629 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027384+0000", 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.23e-07 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027391+0000", 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0899999999999999e-06 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027408+0000", 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.717e-05 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027416+0000", 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1570000000000003e-06 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027791+0000", 2024-09-11T09:29:52.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00037519499999999999 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027902+0000", 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111215 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053383+0000", 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025480860000000001 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:638 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6949973749999998, 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025900833000000002, 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 638 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027489+0000", 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0299999999999998e-07 2024-09-11T09:29:52.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027492+0000", 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6270000000000002e-06 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027501+0000", 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8440000000000004e-06 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027507+0000", 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0689999999999998e-06 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028168+0000", 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066121099999999998 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028276+0000", 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107435 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053389+0000", 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025112944000000002 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:660 3.12 3:4f6d0c71:::rbd_data.1106b643ff7f.000000000000005c:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.694340188, 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025370475, 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.783 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 660 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028146+0000", 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2900000000000004e-07 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028149+0000", 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9000000000000002e-06 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028156+0000", 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2200000000000003e-06 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028161+0000", 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6910000000000002e-06 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028580+0000", 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000418625 2024-09-11T09:29:52.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028685+0000", 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010467300000000001 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053516+0000", 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024830837000000001 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:973 3.1f 3:ff421b5d:::rbd_data.1106b643ff7f.000000000000008d:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.054727+0000", 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6677580519999999, 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024593184000000001, 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 973 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054727+0000", 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054727+0000", 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054728+0000", 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6000000000000004e-07 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054730+0000", 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.43e-06 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054737+0000", 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4479999999999999e-06 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054742+0000", 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1560000000000003e-06 2024-09-11T09:29:52.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055595+0000", 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00085359900000000002 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055680+0000", 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4019999999999999e-05 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079320+0000", 2024-09-11T09:29:52.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.023640971 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1000 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6675032859999996, 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037281880000000003, 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.792 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1000 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054983+0000", 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054987+0000", 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1670000000000001e-06 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054996+0000", 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7490000000000004e-06 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055002+0000", 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4799999999999998e-06 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055397+0000", 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00039521700000000001 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055474+0000", 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6872000000000005e-05 2024-09-11T09:29:52.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055665+0000", 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000191013 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056748+0000", 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010830779999999999 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092163+0000", 2024-09-11T09:29:52.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035415042000000001 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092225+0000", 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1555000000000003e-05 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092264+0000", 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9020999999999998e-05 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.797 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1010 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6673463349999995, 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037320416000000002, 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1010 2024-09-11T09:29:52.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:29:52.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2199999999999999e-07 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055142+0000", 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5620000000000002e-06 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055148+0000", 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2400000000000004e-06 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055153+0000", 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.977e-06 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056168+0000", 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010149040000000001 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056237+0000", 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8606e-05 2024-09-11T09:29:52.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056413+0000", 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017639199999999999 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057731+0000", 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001317616 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092370+0000", 2024-09-11T09:29:52.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034638872000000001 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092419+0000", 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8989999999999997e-05 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092459+0000", 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0834999999999998e-05 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1011 3.1f 3:fb0b56ac:::rbd_data.1106b643ff7f.0000000000000048:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.055160+0000", 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.6673252429999996, 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024720552, 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1011 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055160+0000", 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055160+0000", 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055161+0000", 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4700000000000002e-07 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055163+0000", 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7630000000000001e-06 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055171+0000", 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3050000000000004e-06 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055176+0000", 2024-09-11T09:29:52.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.933e-06 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057813+0000", 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0026371580000000001 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057920+0000", 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107329 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.058116+0000", 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00019628500000000001 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.058691+0000", 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000574381 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079757+0000", 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.021066685000000002 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079827+0000", 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0098999999999994e-05 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.079881+0000", 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3167000000000001e-05 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.809 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1018 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.665497877, 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035657162999999999, 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1018 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.810 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056988+0000", 2024-09-11T09:29:52.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.95e-07 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056994+0000", 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4500000000000003e-06 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057005+0000", 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1522000000000001e-05 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057013+0000", 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6599999999999995e-06 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057058+0000", 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057153+0000", 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5377000000000005e-05 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057338+0000", 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018502199999999999 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057925+0000", 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00058726299999999998 2024-09-11T09:29:52.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092552+0000", 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034627003000000003 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092598+0000", 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5989000000000003e-05 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092645+0000", 2024-09-11T09:29:52.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6431999999999999e-05 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1086 3.12 3:4b95dab8:::rbd_data.1106b643ff7f.0000000000000085:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:29:52.816 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.661844372, 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.032140305000000001, 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1086 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:52.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060642+0000", 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4300000000000002e-07 2024-09-11T09:29:52.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060650+0000", 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1969999999999996e-06 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060660+0000", 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0471e-05 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:52.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060667+0000", 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4829999999999996e-06 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060798+0000", 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013138599999999999 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060884+0000", 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5348999999999999e-05 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061065+0000", 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018090700000000001 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061845+0000", 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00078057400000000002 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092700+0000", 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.030854940000000001 2024-09-11T09:29:52.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092749+0000", 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9101999999999998e-05 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092781+0000", 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2053e-05 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:52.822 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:52.823 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:52.824 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:52.825 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:29:52.826 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:52.825+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:52.925 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:52.925+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:52.973 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:52.973 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:52.973 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:52.973 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:52.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:53.025 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:53.026+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:53.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:53.126 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:53.126+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:53.137 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:53.137 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:53.137 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:53.137 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:53.137 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:53.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:53.227 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:53.227+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:53.238 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:53.239 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:53.243 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone25 2024-09-11T09:29:53.263 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:53.263 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:53.281 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:53.281 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:53.292 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:53.292 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:53.300 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:53.300 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:53.304 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:53.319 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:53.320 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:53.327 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:53.328+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:53.338 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:53.338 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:53.429 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:53.429+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:53.440 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:53.440 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:53.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:53.534 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:53.535+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:53.598 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-11T09:29:53.611 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:53.611 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:53.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:53.663 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:53.664+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:53.732 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:53.732+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:53.781 INFO:teuthology.orchestra.run.smithi107.stdout:4001 read 0xdf0200 thru 0xdf4de2 (0x4be3 bytes) 2024-09-11T09:29:53.788 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:53.788 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:53.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:53.839 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:53.839+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:53.902 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:53.902 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:53.911 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:53.954 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:53.955+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:54.013 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.013 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.021 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:54.062 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:54.063+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:54.136 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:54.136+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:54.164 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.164 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:54.256 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:54.257+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:54.316 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.316 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:54.365 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:54.366+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:54.438 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:54.438+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:54.466 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.467 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.475 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:54.540 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:54.540+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:54.577 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.577 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.585 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:54.640 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:54.641+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:54.688 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.688 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:54.741 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:54.741+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:54.794 INFO:teuthology.orchestra.run.smithi107.stdout:4003 trunc from 0xa3cba00 to 0xc5f4000 2024-09-11T09:29:54.817 INFO:teuthology.orchestra.run.smithi107.stdout:4004 read 0x9d97800 thru 0x9d97966 (0x167 bytes) 2024-09-11T09:29:54.817 INFO:teuthology.orchestra.run.smithi107.stdout:4005 read 0xaaeb200 thru 0xaaf4ef1 (0x9cf2 bytes) 2024-09-11T09:29:54.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:29:54.833 INFO:teuthology.orchestra.run.smithi107.stdout:4008 trunc from 0xc5f4000 to 0x1098400 2024-09-11T09:29:54.842 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:54.841+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:54.852 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:54.852 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:54.861 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:54.868 INFO:teuthology.orchestra.run.smithi107.stdout:4012 write 0x7f4dc00 thru 0x7f5b6d2 (0xdad3 bytes) 2024-09-11T09:29:54.869 INFO:teuthology.orchestra.run.smithi107.stdout:4017 punch from 0x54c8a00 to 0x54cb000, (0x2600 bytes) 2024-09-11T09:29:54.870 INFO:teuthology.orchestra.run.smithi107.stdout:4018 write 0x15b4200 thru 0x15c0ee4 (0xcce5 bytes) 2024-09-11T09:29:54.870 INFO:teuthology.orchestra.run.smithi107.stdout:4019 write 0x26ae00 thru 0x2781a5 (0xd3a6 bytes) 2024-09-11T09:29:54.871 INFO:teuthology.orchestra.run.smithi107.stdout:4023 read 0x2136e00 thru 0x213d29e (0x649f bytes) 2024-09-11T09:29:54.871 INFO:teuthology.orchestra.run.smithi107.stdout:4026 write 0x6909800 thru 0x690bb06 (0x2307 bytes) 2024-09-11T09:29:54.871 INFO:teuthology.orchestra.run.smithi107.stdout:4027 punch from 0x6f6b000 to 0x6f6f200, (0x4200 bytes) 2024-09-11T09:29:54.876 INFO:teuthology.orchestra.run.smithi107.stdout:4028 read 0x1117600 thru 0x111e637 (0x7038 bytes) 2024-09-11T09:29:54.882 INFO:teuthology.orchestra.run.smithi107.stdout:4029 read 0x84a00 thru 0x8f666 (0xac67 bytes) 2024-09-11T09:29:54.902 INFO:teuthology.orchestra.run.smithi107.stdout:4031 write 0x49cf400 thru 0x49db9e2 (0xc5e3 bytes) 2024-09-11T09:29:54.902 INFO:teuthology.orchestra.run.smithi107.stdout:4037 punch from 0x632000 to 0x636600, (0x4600 bytes) 2024-09-11T09:29:54.942 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:54.942+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:55.006 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:55.006 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:55.015 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:55.058 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:55.058+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:55.123 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:55.123 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:55.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:55.172 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:55.172+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:55.245 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:55.245+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:55.279 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:55.279 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:55.287 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:55.346 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:55.346+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:55.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:55.394 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:55.395 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:55.404 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:55.446 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:55.446+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:55.476 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:55.476 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:55.493 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:55.494 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:55.510 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:55.528 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:55.528 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:55.546 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:55.547 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:55.546+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:55.553 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:55.553 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:55.553 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:55.553 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:55.563 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:55.563 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:55.564 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:55.647 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:55.647+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:55.670 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:55.670 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:55.670 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:55.670 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:55.670 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:55.679 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:55.748 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:55.749+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:55.778 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:29:55.778 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:29:55.778 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-11T09:29:55.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-11T09:29:55.783 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:29:55.835 INFO:teuthology.orchestra.run.smithi107.stdout:4041 write 0xb148400 thru 0xb14e1ba (0x5dbb bytes) 2024-09-11T09:29:55.842 INFO:teuthology.orchestra.run.smithi107.stdout:4042 write 0x9737800 thru 0x9741031 (0x9832 bytes) 2024-09-11T09:29:55.844 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:55.844 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:55.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:55.844 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:55.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.844 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.933023624, 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:29:55.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:29:55.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:55.849 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.932793052000001, 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:55.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:29:55.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.932695848000002, 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.854 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:29:55.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:29:55.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.932461084000003, 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:29:55.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:29:55.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:55.863 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.932265068, 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.864 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:29:55.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.868 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.931996630999997, 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:29:55.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.931894880999998, 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:29:55.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.931501224000002, 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:29:55.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.931374978999997, 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:29:55.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:29:55.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.931165597000003, 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:55.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:29:55.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.892 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.931054648, 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.894 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:55.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:29:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:29:55.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:29:55.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:29:55.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:29:55.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.899 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:29:55.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:29:55.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:29:55.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:29:55.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:29:55.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:29:55.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.901 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:29:55.902 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:29:55.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.902 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.903 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.903 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:55.903 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.930798709999998, 2024-09-11T09:29:55.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:29:55.903 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.903 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:29:55.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.905 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:55.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.907 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:29:55.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:29:55.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:29:55.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:29:55.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:29:55.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:29:55.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:29:55.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.911 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.930646181, 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.912 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:29:55.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.930453817, 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:29:55.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:29:55.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.930123399000003, 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.923 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.929847068999997, 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.928 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.929 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:29:55.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.932 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:29:55.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.934 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.935 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.929793148999998, 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.936 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:29:55.938 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.939 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:29:55.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.929592579999998, 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.942 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.929237041, 2024-09-11T09:29:55.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.947 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.950 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 13.149971508, 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:29:55.951 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.952 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:29:55.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:29:55.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:29:55.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:55.957 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:55.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:55.961 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-11T09:29:55.961 INFO:teuthology.orchestra.run.smithi107.stdout:4043 trunc from 0xb14e200 to 0x87c2800 2024-09-11T09:29:55.961 INFO:teuthology.orchestra.run.smithi107.stdout:4044 read 0x5785400 thru 0x5790ee2 (0xbae3 bytes) 2024-09-11T09:29:55.961 INFO:teuthology.orchestra.run.smithi107.stdout:4050 read 0x7c38a00 thru 0x7c48860 (0xfe61 bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4051 read 0x6e66600 thru 0x6e6fabc (0x94bd bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4052 write 0x9be2c00 thru 0x9bebd12 (0x9113 bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4053 write 0x628000 thru 0x6327cb (0xa7cc bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4054 read 0x16acc00 thru 0x16b23cc (0x57cd bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4059 read 0x73dd800 thru 0x73e4d37 (0x7538 bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4060 write 0x8005a00 thru 0x800f5e1 (0x9be2 bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4061 punch from 0x7405200 to 0x7407e00, (0x2c00 bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4062 read 0x7c8de00 thru 0x7c99d53 (0xbf54 bytes) 2024-09-11T09:29:55.962 INFO:teuthology.orchestra.run.smithi107.stdout:4064 clone 27 order 22 su 262144 sc 9 2024-09-11T09:29:55.963 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:55.959+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:56.059 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:56.059+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:56.110 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:56.110 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:56.110 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:29:56.110 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:56.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:56.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-11T09:29:56.180 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:56.180+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:56.228 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:56.228 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:29:56.228 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:29:56.228 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:29:56.229 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:56.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:56.261 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:56.261+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:56.340 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-11T09:29:56.362 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:56.361+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:56.366 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:13 3.1e 3:7ded0e34:::rbd_object_map.11065aee08ca:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.644328905999998, 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013658920000000001, 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.367 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 13 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721825+0000", 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721827+0000", 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7430000000000001e-06 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721834+0000", 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5950000000000004e-06 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721846+0000", 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2241e-05 2024-09-11T09:29:56.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721863+0000", 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7110000000000001e-05 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.721918+0000", 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5251999999999999e-05 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722013+0000", 2024-09-11T09:29:56.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4958999999999997e-05 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722147+0000", 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000133409 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.722813+0000", 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066613000000000002 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723132+0000", 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00031880999999999999 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723166+0000", 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4722999999999999e-05 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:28:48.723191+0000", 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4919999999999999e-05 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.372 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPush(3.7 74/73 [PushOp(3:e7934249:::rbd_data.11062c268ffd.0000000000000005:head, version: 29'13, data_included: [0~491520], data_size: 491520, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:e7934249:::rbd_data.11062c268ffd.0000000000000005:head@29'13, size: 491520, copy_subset: [(0, 491520)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 491520, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.147766+0000", 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.2183880650000001, 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011198099999999999, 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.147766+0000", 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.147766+0000", 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.147767+0000", 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7999999999999993e-07 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.148153+0000", 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00038599800000000003 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.148178+0000", 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5343000000000001e-05 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.148200+0000", 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1064e-05 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.148306+0000", 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000106804 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.148316+0000", 2024-09-11T09:29:56.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.268e-06 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.148886+0000", 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00057035300000000001 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.5323.0:832 3.18 e74/73 3:18760478:::rbd_header.1106a471d92f:head v 74'296, mlcod=0'0)", 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.158292+0000", 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.2078621140000001, 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001258353, 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158292+0000", 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158292+0000", 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158292+0000", 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1100000000000002e-07 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158299+0000", 2024-09-11T09:29:56.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4169999999999997e-06 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158315+0000", 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6113e-05 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158326+0000", 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0791999999999999e-05 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158396+0000", 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9814000000000006e-05 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158422+0000", 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6279999999999999e-05 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159501+0000", 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010796099999999999 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159550+0000", 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9016e-05 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "pg_update_log_missing(3.18 epoch 74/73 rep_tid 2113 entries 74'297 (0'0) error 3:18760478:::rbd_header.1106a471d92f:head by client.5323.0:833 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 28'3)", 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.160400+0000", 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.2057539080000002, 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00087620700000000005, 2024-09-11T09:29:56.382 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "reached pg", 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160400+0000", 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160400+0000", 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160401+0000", 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0900000000000001e-07 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160411+0000", 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5389999999999996e-06 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160431+0000", 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0729000000000001e-05 2024-09-11T09:29:56.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160447+0000", 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5296000000000001e-05 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160521+0000", 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4375000000000005e-05 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.161277+0000", 2024-09-11T09:29:56.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00075555900000000005 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4358.0:16062 3.18 e74/73 3:18760478:::rbd_header.1106a471d92f:head v 74'299, mlcod=28'3)", 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.174666+0000", 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.1914882370000002, 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00078091900000000004, 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174666+0000", 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174666+0000", 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174667+0000", 2024-09-11T09:29:56.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.61e-07 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174676+0000", 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7749999999999998e-06 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174688+0000", 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.2853e-05 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174696+0000", 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6210000000000004e-06 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174759+0000", 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2934000000000004e-05 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174777+0000", 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7609e-05 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.175399+0000", 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00062206299999999996 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.175447+0000", 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8303000000000002e-05 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPush(3.7 74/73 [PushOp(3:e602d124:::rbd_data.1106cda8c42d.0000000000000000:a, version: 36'59, data_included: [0~299008], data_size: 299008, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:e602d124:::rbd_data.1106cda8c42d.0000000000000000:a@36'59, size: 299008, copy_subset: [(0, 299008)], clone_subset: {}, snapset: a=[]:{a: [a]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 299008, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.174767+0000", 2024-09-11T09:29:56.390 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.1913871560000002, 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00094753999999999997, 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174767+0000", 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174767+0000", 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174768+0000", 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3900000000000004e-07 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174880+0000", 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111958 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174899+0000", 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9454999999999999e-05 2024-09-11T09:29:56.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.174917+0000", 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7620000000000001e-05 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.175045+0000", 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000128479 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.175070+0000", 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4488e-05 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.175715+0000", 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00064470100000000002 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPush(3.7 74/73 [PushOp(3:e3855b00:::rbd_data.1106551ea4b0.0000000000000001:e, version: 42'75, data_included: [0~786432], data_size: 786432, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:e3855b00:::rbd_data.1106551ea4b0.0000000000000001:e@42'75, size: 3145728, copy_subset: [(0, 3145728)], clone_subset: {}, snapset: e=[]:{e: [e]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3145728, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:29:56.394 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.187998+0000", 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.1781564869999999, 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0013746240000000001, 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.187998+0000", 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.187998+0000", 2024-09-11T09:29:56.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.187998+0000", 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.188244+0000", 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000245348 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.188261+0000", 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7856999999999999e-05 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.188275+0000", 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3176000000000001e-05 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.188342+0000", 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7589000000000006e-05 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.188350+0000", 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6760000000000007e-06 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.189372+0000", 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001022429 2024-09-11T09:29:56.398 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPush(3.7 74/73 [PushOp(3:e3855b00:::rbd_data.1106551ea4b0.0000000000000001:head, version: 42'76, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:e3855b00:::rbd_data.1106551ea4b0.0000000000000001:head@42'76, size: 524288, copy_subset: [], clone_subset: {3:e3855b00:::rbd_data.1106551ea4b0.0000000000000001:e: [(0, 524288)]}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.190670+0000", 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.1754838840000001, 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000777453, 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190670+0000", 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190670+0000", 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190671+0000", 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0100000000000004e-07 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190686+0000", 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4615e-05 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190707+0000", 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1696e-05 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190728+0000", 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0840999999999999e-05 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190826+0000", 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7967999999999994e-05 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.190842+0000", 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5933000000000001e-05 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.191448+0000", 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00060559899999999996 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPush(3.7 74/73 [PushOp(3:e37e43a4:::rbd_data.11068c3dd90a.000000000000000a:head, version: 60'167, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:e37e43a4:::rbd_data.11068c3dd90a.000000000000000a:head@60'167, size: 3670016, copy_subset: [(0, 3670016)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3670016, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.232487+0000", 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.1336667610000002, 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00087767400000000005, 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232487+0000", 2024-09-11T09:29:56.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232487+0000", 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232488+0000", 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6499999999999999e-07 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232716+0000", 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00022804499999999999 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232732+0000", 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5427e-05 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232741+0000", 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9250000000000007e-06 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232814+0000", 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2479999999999997e-05 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.232821+0000", 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9990000000000002e-06 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.233365+0000", 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00054423299999999998 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPush(3.9 74/73 [PushOp(3:94da3b10:::rbd_data.11065aee08ca.000000000000000c:head, version: 25'7, data_included: [503808~65536,3055616~20480], data_size: 86016, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:94da3b10:::rbd_data.11065aee08ca.000000000000000c:head@25'7, size: 3076096, copy_subset: [(0, 3076096)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 3076096, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:53.254085+0000", 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.112069526, 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000906874, 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254085+0000", 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254085+0000", 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254085+0000", 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5099999999999999e-07 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254164+0000", 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8788000000000005e-05 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254187+0000", 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3053000000000001e-05 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254208+0000", 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0336e-05 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254306+0000", 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8771999999999996e-05 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254320+0000", 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4114000000000001e-05 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.254992+0000", 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00067115999999999996 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.5359.0:1056 3.9 e77/75 3:923522a4:::rbd_data.1106c83ffe2c.0000000000000000:head v 77'344, mlcod=76'342)", 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.805210+0000", 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.56094405800000002, 2024-09-11T09:29:56.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.004426981, 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.805210+0000", 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.805210+0000", 2024-09-11T09:29:56.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.805211+0000", 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7199999999999999e-07 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.807461+0000", 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022506269999999998 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.807472+0000", 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0747999999999999e-05 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.807479+0000", 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7320000000000001e-06 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.807535+0000", 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6028e-05 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.807551+0000", 2024-09-11T09:29:56.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6042e-05 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.809596+0000", 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020450939999999999 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.809637+0000", 2024-09-11T09:29:56.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1137999999999998e-05 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1060 3.18 3:181b38c3:::rbd_data.1106c83ffe2c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 634368~5120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.812038+0000", 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.55411618500000004, 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0072405550000000001, 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1060 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.812038+0000", 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.812038+0000", 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.812039+0000", 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8700000000000004e-07 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.814440+0000", 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0024007680000000002 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.814458+0000", 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8470000000000001e-05 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.814468+0000", 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8349999999999999e-06 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.814537+0000", 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8418999999999997e-05 2024-09-11T09:29:56.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.814678+0000", 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014130499999999999 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.814995+0000", 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000317468 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.818333+0000", 2024-09-11T09:29:56.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0033373909999999999 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.819190+0000", 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00085750300000000002 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.819244+0000", 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4082999999999997e-05 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.819279+0000", 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4326000000000003e-05 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1124 3.8 3:1573ae9d:::rbd_data.1106c83ffe2c.000000000000005a:head [set-alloc-hint object_size 2097152 write_size 2097152,write 328704~15360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.856035+0000", 2024-09-11T09:29:56.423 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.51011960199999995, 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021207779999999998, 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1124 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856035+0000", 2024-09-11T09:29:56.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856035+0000", 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856036+0000", 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.139e-06 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856060+0000", 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4156e-05 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856074+0000", 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3791000000000001e-05 2024-09-11T09:29:56.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856084+0000", 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9059999999999997e-06 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856153+0000", 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9598999999999996e-05 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856277+0000", 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000123943 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.856495+0000", 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000217841 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857483+0000", 2024-09-11T09:29:56.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00098833199999999997 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.858069+0000", 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00058576500000000003 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.858116+0000", 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7185999999999999e-05 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.858155+0000", 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9119999999999998e-05 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1135 3.8 3:1573ae9d:::rbd_data.1106c83ffe2c.000000000000005a:head [read 212992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.857678+0000", 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.50847668700000004, 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000786253, 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.430 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1135 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857678+0000", 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857678+0000", 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857678+0000", 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3300000000000002e-07 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857681+0000", 2024-09-11T09:29:56.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4760000000000001e-06 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857688+0000", 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8979999999999999e-06 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857694+0000", 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4740000000000002e-06 2024-09-11T09:29:56.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857799+0000", 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.857834+0000", 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4984999999999998e-05 2024-09-11T09:29:56.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.858366+0000", 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00053202299999999996 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.858403+0000", 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6247e-05 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.858464+0000", 2024-09-11T09:29:56.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1282e-05 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1159 3.8 3:1573ae9d:::rbd_data.1106c83ffe2c.000000000000005a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.862572+0000", 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.503581748, 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017948300000000001, 2024-09-11T09:29:56.436 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.437 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.437 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.437 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.437 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.437 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.437 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1159 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862572+0000", 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862572+0000", 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862574+0000", 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.051e-06 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862579+0000", 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0159999999999999e-06 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862594+0000", 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5082000000000001e-05 2024-09-11T09:29:56.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862602+0000", 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9570000000000002e-06 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862654+0000", 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2833000000000001e-05 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862737+0000", 2024-09-11T09:29:56.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2600000000000002e-05 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.862837+0000", 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010041500000000001 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.863991+0000", 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001153945 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.864315+0000", 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00032340299999999999 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.864345+0000", 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0553999999999999e-05 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.864367+0000", 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1974e-05 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.442 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1263 3.18 3:181b38c3:::rbd_data.1106c83ffe2c.0000000000000002:head [stat,write 1605632~16384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.893639+0000", 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.47251570300000001, 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020158329999999999, 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.443 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1263 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893639+0000", 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893639+0000", 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893640+0000", 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1620000000000001e-06 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893658+0000", 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8255000000000001e-05 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893674+0000", 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5718000000000001e-05 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893682+0000", 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2989999999999997e-06 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893760+0000", 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7595000000000005e-05 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893847+0000", 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7915000000000001e-05 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.893997+0000", 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014997799999999999 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894843+0000", 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00084576400000000004 2024-09-11T09:29:56.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895547+0000", 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00070420499999999996 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895599+0000", 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1743999999999998e-05 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895654+0000", 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5198000000000002e-05 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.5359.0:1262 3.9 e77/75 3:923522a4:::rbd_data.1106c83ffe2c.0000000000000000:head v 77'345, mlcod=76'343)", 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.894432+0000", 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.47172193899999998, 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00101439, 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.449 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894432+0000", 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894432+0000", 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894433+0000", 2024-09-11T09:29:56.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8100000000000001e-07 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894448+0000", 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4895000000000001e-05 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894459+0000", 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1241e-05 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894466+0000", 2024-09-11T09:29:56.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5780000000000004e-06 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894524+0000", 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7751000000000003e-05 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894543+0000", 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9511999999999999e-05 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895402+0000", 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00085872200000000002 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895447+0000", 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4709999999999997e-05 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1277 3.18 3:181b38c3:::rbd_data.1106c83ffe2c.0000000000000002:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.894937+0000", 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.47121753399999999, 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00093937800000000004, 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1277 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.454 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894937+0000", 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894937+0000", 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894938+0000", 2024-09-11T09:29:56.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7599999999999996e-07 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894944+0000", 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5050000000000004e-06 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894954+0000", 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0414000000000001e-05 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.894962+0000", 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0550000000000002e-06 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895016+0000", 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4539000000000001e-05 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895258+0000", 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000241697 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895698+0000", 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00044030799999999999 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895764+0000", 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5968000000000003e-05 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.895876+0000", 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011201600000000001 2024-09-11T09:29:56.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1282 3.4 3:23220615:::rbd_data.1106c83ffe2c.0000000000000041:head [set-alloc-hint object_size 2097152 write_size 2097152,write 6656~9728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.897116+0000", 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.46903841099999999, 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017544310000000001, 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.459 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1282 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897116+0000", 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897116+0000", 2024-09-11T09:29:56.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897117+0000", 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0979999999999999e-06 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897130+0000", 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3560000000000001e-05 2024-09-11T09:29:56.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897145+0000", 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4246999999999999e-05 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897153+0000", 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2020000000000004e-06 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897228+0000", 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5319000000000002e-05 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897351+0000", 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012300500000000001 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.897479+0000", 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00012787000000000001 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.898230+0000", 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00075097300000000005 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.898764+0000", 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00053405 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.898824+0000", 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9806000000000003e-05 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.898870+0000", 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6301000000000003e-05 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5359.0:1305 3.8 3:10228a3d:::rbd_data.1106c83ffe2c.0000000000000039:head [set-alloc-hint object_size 2097152 write_size 2097152,write 1053184~11264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:55.901665+0000", 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.46448928099999998, 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018200040000000001, 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5359", 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4011506646", 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1305 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:29:56.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901665+0000", 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901665+0000", 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901666+0000", 2024-09-11T09:29:56.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4399999999999999e-07 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901678+0000", 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1831000000000001e-05 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901688+0000", 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.049e-05 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901695+0000", 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5139999999999998e-06 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901739+0000", 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4795000000000001e-05 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901812+0000", 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2212000000000006e-05 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.901898+0000", 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6550000000000003e-05 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.903050+0000", 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011518310000000001 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.903388+0000", 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00033820900000000001 2024-09-11T09:29:56.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.903444+0000", 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5964000000000002e-05 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:55.903485+0000", 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0864e-05 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:29:56.471 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:29:56.472 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:29:56.473 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:56.474 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:56.472+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:56.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-11T09:29:56.573 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:56.573+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:56.581 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:56.581 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:56.581 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:56.581 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:56.589 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:56.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-11T09:29:56.698 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:56.698+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:56.732 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:56.732 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:56.741 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:56.784 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:56.784+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:56.855 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.855 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.417096366999999, 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:29:56.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:29:56.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:29:56.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:29:56.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:29:56.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.862 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8011906399999997, 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:29:56.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:29:56.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8009929390000003, 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:29:56.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:29:56.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:29:56.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:56.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.875 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8007345130000001, 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:29:56.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:29:56.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8007046290000002, 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.882 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:29:56.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.8001714769999992, 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:29:56.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7992940280000003, 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:29:56.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7992550620000003, 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:29:56.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7992115860000002, 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:29:56.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:56.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7990869689999993, 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7985214989999996, 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:29:56.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7984908629999996, 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:56.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:29:56.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:29:56.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:56.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7963544309999993, 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:29:56.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7948339430000004, 2024-09-11T09:29:56.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:56.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:29:56.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7907601139999993, 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:56.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:29:56.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7906463200000005, 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.935 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:29:56.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.940 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7905616379999998, 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:29:56.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:29:56.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:29:56.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7905156469999994, 2024-09-11T09:29:56.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:29:56.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7901443490000002, 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:29:56.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.7900024000000005, 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:29:56.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:56.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:56.960 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:56.961 INFO:teuthology.orchestra.run.smithi107.stdout:leaving image image_client.0-clone26 intact 2024-09-11T09:29:56.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:56.963 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:56.961+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:57.062 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:57.062+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:57.108 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:57.109 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:57.109 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:57.109 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:57.118 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:57.202 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:57.202+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:57.222 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-11T09:29:57.263 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:29:57.263 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:29:57.263 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:57.264 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:57.264+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:57.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:29:57.364 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:57.364+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:57.426 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:57.426 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058861630000003, 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058824244999997, 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:29:57.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:29:57.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058804926999997, 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:29:57.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:29:57.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058793680999997, 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:29:57.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.446 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058751776000001, 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:29:57.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:29:57.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058734372000004, 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:29:57.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:29:57.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058716476999997, 2024-09-11T09:29:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:29:57.457 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:29:57.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:29:57.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:29:57.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058696296999997, 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:29:57.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:29:57.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058675313000002, 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.468 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:29:57.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:29:57.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:29:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:29:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:29:57.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.473 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058648216000002, 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:29:57.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:29:57.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.478 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058628929000001, 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:57.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:29:57.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.058424569000003, 2024-09-11T09:29:57.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:29:57.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057890813, 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:57.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057841920000001, 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:29:57.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:29:57.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.497 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057793832000002, 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:29:57.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.502 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057755794000002, 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:29:57.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057531951999998, 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:29:57.507 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:29:57.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057354850000003, 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.512 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:57.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:29:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:29:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:29:57.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057209295, 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:29:57.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.522 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.057087909000003, 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:29:57.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:29:57.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:29:57.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:29:57.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:29:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:29:57.528 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:29:57.528 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:57.530 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:29:57.530 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:57.529+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:57.575 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:29:57.629 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:57.629+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:57.655 INFO:teuthology.orchestra.run.smithi107.stdout:4067 punch from 0xdf6e00 to 0xdfc000, (0x5200 bytes) 2024-09-11T09:29:57.675 INFO:teuthology.orchestra.run.smithi107.stdout:4068 read 0x1eea400 thru 0x1ef7e38 (0xda39 bytes) 2024-09-11T09:29:57.675 INFO:teuthology.orchestra.run.smithi107.stdout:4071 punch from 0x96bd600 to 0x96c7e00, (0xa800 bytes) 2024-09-11T09:29:57.677 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:29:57.678 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:29:57.678 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:29:57.678 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:29:57.680 INFO:teuthology.orchestra.run.smithi107.stdout:4073 read 0x6f61200 thru 0x6f66828 (0x5629 bytes) 2024-09-11T09:29:57.681 INFO:teuthology.orchestra.run.smithi107.stdout:4076 trunc from 0x9bebe00 to 0x8ad0000 2024-09-11T09:29:57.688 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:29:57.708 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:57.708 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:57.723 INFO:teuthology.orchestra.run.smithi107.stdout:4077 trunc from 0x8ad0000 to 0x7e7b800 2024-09-11T09:29:57.727 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:57.728 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:57.732 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:57.732+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:57.745 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:57.745 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:57.756 INFO:teuthology.orchestra.run.smithi107.stdout:4078 write 0xde1d600 thru 0xde2592b (0x832c bytes) 2024-09-11T09:29:57.757 INFO:teuthology.orchestra.run.smithi107.stdout:4079 trunc from 0xde25a00 to 0xd572a00 2024-09-11T09:29:57.767 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:29:57.767 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:29:57.786 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:29:57.786 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:29:57.790 INFO:teuthology.orchestra.run.smithi107.stdout:4080 write 0xda7b400 thru 0xda8873c (0xd33d bytes) 2024-09-11T09:29:57.796 INFO:teuthology.orchestra.run.smithi107.stdout:4081 read 0x766be00 thru 0x766f2d1 (0x34d2 bytes) 2024-09-11T09:29:57.796 INFO:teuthology.orchestra.run.smithi107.stdout:4082 write 0x82e8600 thru 0x82efc93 (0x7694 bytes) 2024-09-11T09:29:57.796 INFO:teuthology.orchestra.run.smithi107.stdout:4083 punch from 0xafb600 to 0xb09400, (0xde00 bytes) 2024-09-11T09:29:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:4085 read 0x955c600 thru 0x955c6d0 (0xd1 bytes) 2024-09-11T09:29:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:4086 read 0x3951800 thru 0x395b762 (0x9f63 bytes) 2024-09-11T09:29:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:4087 read 0x9499400 thru 0x94a5070 (0xbc71 bytes) 2024-09-11T09:29:57.798 INFO:teuthology.orchestra.run.smithi107.stdout:4088 trunc from 0xda88800 to 0xdec1000 2024-09-11T09:29:57.806 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:29:57.806 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:29:57.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:29:57.810 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:57.811 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:57.820 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:29:57.824 INFO:teuthology.orchestra.run.smithi107.stdout:4091 write 0x1e09c00 thru 0x1e0f75a (0x5b5b bytes) 2024-09-11T09:29:57.828 INFO:teuthology.orchestra.run.smithi107.stdout:4094 write 0x9266800 thru 0x927335b (0xcb5c bytes) 2024-09-11T09:29:57.830 INFO:teuthology.orchestra.run.smithi107.stdout:4095 trunc from 0xdec1000 to 0xed08400 2024-09-11T09:29:57.852 INFO:teuthology.orchestra.run.smithi107.stdout:4096 read 0x4a1da00 thru 0x4a2d157 (0xf758 bytes) 2024-09-11T09:29:57.853 INFO:teuthology.orchestra.run.smithi107.stdout:4097 punch from 0x21ea600 to 0x21f4a00, (0xa400 bytes) 2024-09-11T09:29:57.858 INFO:teuthology.orchestra.run.smithi107.stdout:4098 trunc from 0xed08400 to 0xbc69400 2024-09-11T09:29:57.861 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:29:57.861+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:29:57.884 INFO:teuthology.orchestra.run.smithi107.stdout:4099 read 0x156e600 thru 0x157711e (0x8b1f bytes) 2024-09-11T09:29:57.886 INFO:teuthology.orchestra.run.smithi107.stdout:4101 trunc from 0xbc69400 to 0xb5b8200 2024-09-11T09:29:57.897 INFO:teuthology.orchestra.run.smithi107.stdout:4102 punch from 0x78c0400 to 0x78c7600, (0x7200 bytes) 2024-09-11T09:29:57.903 INFO:teuthology.orchestra.run.smithi107.stdout:4103 write 0x932ba00 thru 0x933af87 (0xf588 bytes) 2024-09-11T09:29:57.904 INFO:teuthology.orchestra.run.smithi107.stdout:4109 punch from 0x69e2400 to 0x69e6000, (0x3c00 bytes) 2024-09-11T09:29:57.904 INFO:teuthology.orchestra.run.smithi107.stdout:4112 write 0x877ec00 thru 0x87818b6 (0x2cb7 bytes) 2024-09-11T09:29:57.912 INFO:teuthology.orchestra.run.smithi107.stdout:4114 trunc from 0xb5b8200 to 0x559d800 2024-09-11T09:29:57.931 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:57.930+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:57.948 INFO:teuthology.orchestra.run.smithi107.stdout:4117 trunc from 0x559d800 to 0x846f000 2024-09-11T09:29:57.962 INFO:teuthology.orchestra.run.smithi107.stdout:4119 punch from 0x12fc800 to 0x1309000, (0xc800 bytes) 2024-09-11T09:29:57.962 INFO:teuthology.orchestra.run.smithi107.stdout:4121 read 0xb27c00 thru 0xb2d1d4 (0x55d5 bytes) 2024-09-11T09:29:57.963 INFO:teuthology.orchestra.run.smithi107.stdout:4123 read 0x79dea00 thru 0x79ee8db (0xfedc bytes) 2024-09-11T09:29:57.976 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:57.976 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:57.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:29:57.986 INFO:teuthology.orchestra.run.smithi107.stdout:4124 write 0x9cac800 thru 0x9cb1732 (0x4f33 bytes) 2024-09-11T09:29:57.988 INFO:teuthology.orchestra.run.smithi107.stdout:4130 punch from 0x808a200 to 0x8095800, (0xb600 bytes) 2024-09-11T09:29:57.990 INFO:teuthology.orchestra.run.smithi107.stdout:4134 read 0x73ce600 thru 0x73d6687 (0x8088 bytes) 2024-09-11T09:29:57.990 INFO:teuthology.orchestra.run.smithi107.stdout:4135 read 0x97fa600 thru 0x98073f1 (0xcdf2 bytes) 2024-09-11T09:29:57.992 INFO:teuthology.orchestra.run.smithi107.stdout:4139 punch from 0x7bdaa00 to 0x7be2a00, (0x8000 bytes) 2024-09-11T09:29:57.992 INFO:teuthology.orchestra.run.smithi107.stdout:4140 read 0x9159400 thru 0x9162c78 (0x9879 bytes) 2024-09-11T09:29:57.995 INFO:teuthology.orchestra.run.smithi107.stdout:4141 trunc from 0x9cb1800 to 0x878e200 2024-09-11T09:29:58.031 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:58.031+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:58.038 INFO:teuthology.orchestra.run.smithi107.stdout:4144 write 0xd1caa00 thru 0xd1cc388 (0x1989 bytes) 2024-09-11T09:29:58.038 INFO:teuthology.orchestra.run.smithi107.stdout:4145 read 0x135e400 thru 0x135fe65 (0x1a66 bytes) 2024-09-11T09:29:58.038 INFO:teuthology.orchestra.run.smithi107.stdout:4146 read 0x56cf400 thru 0x56d9b8a (0xa78b bytes) 2024-09-11T09:29:58.038 INFO:teuthology.orchestra.run.smithi107.stdout:4148 punch from 0x8c9a00 to 0x8d1600, (0x7c00 bytes) 2024-09-11T09:29:58.042 INFO:teuthology.orchestra.run.smithi107.stdout:4149 trunc from 0xd1cc400 to 0xab64000 2024-09-11T09:29:58.064 INFO:teuthology.orchestra.run.smithi107.stdout:4151 read 0xd90400 thru 0xd96f06 (0x6b07 bytes) 2024-09-11T09:29:58.064 INFO:teuthology.orchestra.run.smithi107.stdout:4152 clone 28 order 21 su 131072 sc 10 2024-09-11T09:29:58.101 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.102 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:29:58.133 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:58.133+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:58.216 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.217 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.225 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:29:58.233 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:58.234+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:58.332 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.332 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.334 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:58.334+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:58.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:29:58.435 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:58.435+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:58.489 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.490 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.499 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:29:58.540 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:58.540+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:58.637 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:58.637+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:58.646 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.646 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.655 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:29:58.739 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:58.739+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:58.795 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:29:58.810 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.810 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.819 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:29:58.860 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:58.859+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:58.880 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone27 from 0xab64000 (overlap 0x559d800) to 0x2acec00 2024-09-11T09:29:58.940 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:58.940+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:58.978 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:58.978 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:58.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:29:59.041 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:29:59.040+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:29:59.095 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.095 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:29:59.145 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:59.145+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:59.232 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-11T09:29:59.242 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:29:59.241+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:29:59.255 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.255 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:29:59.343 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:59.342+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:59.399 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.399 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:29:59.444 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:29:59.444+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:29:59.540 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.540 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.545 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:59.544+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:59.549 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:29:59.646 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:59.645+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:59.699 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.700 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.708 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:29:59.751 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:59.751+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:59.786 INFO:teuthology.orchestra.run.smithi107.stdout:4153 write 0x6f7e000 thru 0x6f7fdaf (0x1db0 bytes) 2024-09-11T09:29:59.790 INFO:teuthology.orchestra.run.smithi107.stdout:4154 trunc from 0xab64000 to 0xdf29c00 2024-09-11T09:29:59.808 INFO:teuthology.orchestra.run.smithi107.stdout:4156 write 0x1a7aa00 thru 0x1a7e0b1 (0x36b2 bytes) 2024-09-11T09:29:59.814 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.814 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.815 INFO:teuthology.orchestra.run.smithi107.stdout:4157 trunc from 0xdf29c00 to 0x8487400 2024-09-11T09:29:59.819 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:29:59.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:29:59.848 INFO:teuthology.orchestra.run.smithi107.stdout:4159 trunc from 0x8487400 to 0xb902800 2024-09-11T09:29:59.858 INFO:teuthology.orchestra.run.smithi107.stdout:4160 read 0xb678600 thru 0xb67fb2d (0x752e bytes) 2024-09-11T09:29:59.859 INFO:teuthology.orchestra.run.smithi107.stdout:4163 write 0x66d6000 thru 0x66e3f52 (0xdf53 bytes) 2024-09-11T09:29:59.859 INFO:teuthology.orchestra.run.smithi107.stdout:4164 read 0xa5da800 thru 0xa5dd2ad (0x2aae bytes) 2024-09-11T09:29:59.859 INFO:teuthology.orchestra.run.smithi107.stdout:4167 read 0x50af200 thru 0x50bc660 (0xd461 bytes) 2024-09-11T09:29:59.860 INFO:teuthology.orchestra.run.smithi107.stdout:4168 read 0x7082c00 thru 0x7087e57 (0x5258 bytes) 2024-09-11T09:29:59.860 INFO:teuthology.orchestra.run.smithi107.stdout:4169 read 0x96c8800 thru 0x96d77c2 (0xefc3 bytes) 2024-09-11T09:29:59.860 INFO:teuthology.orchestra.run.smithi107.stdout:4170 punch from 0x8abb600 to 0x8acb000, (0xfa00 bytes) 2024-09-11T09:29:59.862 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:29:59.862+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:29:59.873 INFO:teuthology.orchestra.run.smithi107.stdout:4173 trunc from 0xb902800 to 0x3b1e400 2024-09-11T09:29:59.913 INFO:teuthology.orchestra.run.smithi107.stdout:4174 read 0xd46000 thru 0xd4e038 (0x8039 bytes) 2024-09-11T09:29:59.913 INFO:teuthology.orchestra.run.smithi107.stdout:4175 read 0x2733e00 thru 0x273e2d1 (0xa4d2 bytes) 2024-09-11T09:29:59.913 INFO:teuthology.orchestra.run.smithi107.stdout:4177 read 0x2cea00 thru 0x2db522 (0xcb23 bytes) 2024-09-11T09:29:59.914 INFO:teuthology.orchestra.run.smithi107.stdout:4179 read 0x2d0be00 thru 0x2d1320d (0x740e bytes) 2024-09-11T09:29:59.917 INFO:teuthology.orchestra.run.smithi107.stdout:4181 trunc from 0x3b1e400 to 0x14e3400 2024-09-11T09:29:59.949 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:29:59.948+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:29:59.951 INFO:teuthology.orchestra.run.smithi107.stdout:4184 read 0x10f8600 thru 0x10ffab7 (0x74b8 bytes) 2024-09-11T09:29:59.951 INFO:teuthology.orchestra.run.smithi107.stdout:4185 punch from 0x393800 to 0x397a00, (0x4200 bytes) 2024-09-11T09:29:59.953 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:29:59.953 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:29:59.971 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:29:59.971 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:29:59.980 INFO:teuthology.orchestra.run.smithi107.stdout:4188 write 0xce1cc00 thru 0xce25121 (0x8522 bytes) 2024-09-11T09:29:59.980 INFO:teuthology.orchestra.run.smithi107.stdout:4194 read 0x8522000 thru 0x85232c8 (0x12c9 bytes) 2024-09-11T09:29:59.980 INFO:teuthology.orchestra.run.smithi107.stdout:4195 write 0xb5de00 thru 0xb6c1ea (0xe3eb bytes) 2024-09-11T09:29:59.981 INFO:teuthology.orchestra.run.smithi107.stdout:4196 read 0x2114400 thru 0x211eebd (0xaabe bytes) 2024-09-11T09:29:59.981 INFO:teuthology.orchestra.run.smithi107.stdout:4197 read 0x5556a00 thru 0x55630b1 (0xc6b2 bytes) 2024-09-11T09:29:59.984 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:29:59.984 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:29:59.991 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:29:59.991 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:29:59.996 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:00.007 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:00.007 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:00.014 INFO:teuthology.orchestra.run.smithi107.stdout:4198 write 0xd9d9800 thru 0xd9e52f2 (0xbaf3 bytes) 2024-09-11T09:30:00.014 INFO:teuthology.orchestra.run.smithi107.stdout:4199 read 0xab16200 thru 0xab19c3d (0x3a3e bytes) 2024-09-11T09:30:00.024 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:00.024 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:00.026 INFO:teuthology.orchestra.run.smithi107.stdout:4201 trunc from 0xd9e5400 to 0x37dc600 2024-09-11T09:30:00.041 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:00.042 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:00.050 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:00.049+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:00.064 INFO:teuthology.orchestra.run.smithi107.stdout:4202 write 0xb823c00 thru 0xb82ea6c (0xae6d bytes) 2024-09-11T09:30:00.065 INFO:teuthology.orchestra.run.smithi107.stdout:4203 read 0x8132000 thru 0x813a801 (0x8802 bytes) 2024-09-11T09:30:00.065 INFO:teuthology.orchestra.run.smithi107.stdout:4204 write 0x7f99600 thru 0x7f9be82 (0x2883 bytes) 2024-09-11T09:30:00.081 INFO:teuthology.orchestra.run.smithi107.stdout:4205 write 0xdc20400 thru 0xdc28ec3 (0x8ac4 bytes) 2024-09-11T09:30:00.081 INFO:teuthology.orchestra.run.smithi107.stdout:4206 punch from 0x7446000 to 0x744a800, (0x4800 bytes) 2024-09-11T09:30:00.092 INFO:teuthology.orchestra.run.smithi107.stdout:4207 trunc from 0xdc29000 to 0x82dbc00 2024-09-11T09:30:00.112 INFO:teuthology.orchestra.run.smithi107.stdout:4208 write 0x35fc600 thru 0x3608a1f (0xc420 bytes) 2024-09-11T09:30:00.120 INFO:teuthology.orchestra.run.smithi107.stdout:4209 trunc from 0x82dbc00 to 0xe57ec00 2024-09-11T09:30:00.137 INFO:teuthology.orchestra.run.smithi107.stdout:4211 write 0xe00ee00 thru 0xe010e4d (0x204e bytes) 2024-09-11T09:30:00.138 INFO:teuthology.orchestra.run.smithi107.stdout:4212 read 0xb307000 thru 0xb30a2f0 (0x32f1 bytes) 2024-09-11T09:30:00.138 INFO:teuthology.orchestra.run.smithi107.stdout:4214 write 0xc56a00 thru 0xc62b84 (0xc185 bytes) 2024-09-11T09:30:00.139 INFO:teuthology.orchestra.run.smithi107.stdout:4217 punch from 0x454d400 to 0x4551600, (0x4200 bytes) 2024-09-11T09:30:00.139 INFO:teuthology.orchestra.run.smithi107.stdout:4218 read 0x90cb400 thru 0x90d7b79 (0xc77a bytes) 2024-09-11T09:30:00.143 INFO:teuthology.orchestra.run.smithi107.stdout:4219 trunc from 0xe57ec00 to 0xb621800 2024-09-11T09:30:00.151 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:00.150+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:00.158 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790633251000003, 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:30:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790595865999997, 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:30:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790576547999997, 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:30:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790565301999997, 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:30:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:30:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790523397000001, 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:30:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790505992999996, 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790488097999997, 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:30:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:30:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790467917999997, 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:30:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:30:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790446934000002, 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:30:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790419837000002, 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:30:00.206 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790400550000001, 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:30:00.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:30:00.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.790196190000003, 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.789662434, 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:30:00.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.789613541000001, 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:00.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.789565453000002, 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:30:00.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:30:00.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.789527415000002, 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:30:00.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.233 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.789303572999998, 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:30:00.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:30:00.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.789126471000003, 2024-09-11T09:30:00.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:30:00.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.788980916, 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:30:00.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:00.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 56.788859530000003, 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.251 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.252 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:00.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:00.258 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:00.252+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4220 read 0x3f45a00 thru 0x3f4f6b7 (0x9cb8 bytes) 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4221 write 0x2203e00 thru 0x220c704 (0x8905 bytes) 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4224 punch from 0x9c76e00 to 0x9c7b000, (0x4200 bytes) 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4225 read 0x1db9000 thru 0x1dc13a4 (0x83a5 bytes) 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4226 trunc from 0xb621800 to 0xe4e3c00 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4228 write 0xae5a800 thru 0xae683c3 (0xdbc4 bytes) 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4230 read 0xdce5c00 thru 0xdce9d72 (0x4173 bytes) 2024-09-11T09:30:00.258 INFO:teuthology.orchestra.run.smithi107.stdout:4231 write 0xcab7800 thru 0xcac5fe4 (0xe7e5 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4233 write 0x1910c00 thru 0x191f28b (0xe68c bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4234 read 0x7227800 thru 0x7235161 (0xd962 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4235 write 0x5c7ec00 thru 0x5c879fe (0x8dff bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4238 read 0x2899400 thru 0x289b525 (0x2126 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4241 write 0x8a89a00 thru 0x8a960b6 (0xc6b7 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4244 trunc from 0xe4e3c00 to 0x8288200 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4249 read 0x7733000 thru 0x773f578 (0xc579 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4250 write 0x6cf7200 thru 0x6cfd764 (0x6565 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4252 write 0xc964e00 thru 0xc96c531 (0x7732 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4253 read 0x42a3400 thru 0x42a702d (0x3c2e bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4255 punch from 0x9e92400 to 0x9ea2000, (0xfc00 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4256 read 0x9e04000 thru 0x9e10add (0xcade bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4257 write 0x42e4e00 thru 0x42eb730 (0x6931 bytes) 2024-09-11T09:30:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:4261 read 0xa921200 thru 0xa922a01 (0x1802 bytes) 2024-09-11T09:30:00.260 INFO:teuthology.orchestra.run.smithi107.stdout:4264 trunc from 0xc96c600 to 0x2b16600 2024-09-11T09:30:00.299 INFO:teuthology.orchestra.run.smithi107.stdout:4266 write 0x9e18c00 thru 0x9e1eacd (0x5ece bytes) 2024-09-11T09:30:00.299 INFO:teuthology.orchestra.run.smithi107.stdout:4270 read 0x8a5b200 thru 0x8a60253 (0x5054 bytes) 2024-09-11T09:30:00.300 INFO:teuthology.orchestra.run.smithi107.stdout:4271 read 0x4497e00 thru 0x44a3c07 (0xbe08 bytes) 2024-09-11T09:30:00.300 INFO:teuthology.orchestra.run.smithi107.stdout:4272 read 0x277be00 thru 0x2783f38 (0x8139 bytes) 2024-09-11T09:30:00.300 INFO:teuthology.orchestra.run.smithi107.stdout:4273 write 0x54be000 thru 0x54c9fbb (0xbfbc bytes) 2024-09-11T09:30:00.300 INFO:teuthology.orchestra.run.smithi107.stdout:4277 write 0x3a6e000 thru 0x3a70e82 (0x2e83 bytes) 2024-09-11T09:30:00.301 INFO:teuthology.orchestra.run.smithi107.stdout:4279 punch from 0x3bb0800 to 0x3bbfc00, (0xf400 bytes) 2024-09-11T09:30:00.304 INFO:teuthology.orchestra.run.smithi107.stdout:4280 trunc from 0x9e1ec00 to 0x70ce600 2024-09-11T09:30:00.326 INFO:teuthology.orchestra.run.smithi107.stdout:4282 read 0x48f5400 thru 0x48ffe1b (0xaa1c bytes) 2024-09-11T09:30:00.331 INFO:teuthology.orchestra.run.smithi107.stdout:4283 trunc from 0x70ce600 to 0xaad9000 2024-09-11T09:30:00.348 INFO:teuthology.orchestra.run.smithi107.stdout:4284 trunc from 0xaad9000 to 0x48f4800 2024-09-11T09:30:00.354 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:00.354+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:00.365 INFO:teuthology.orchestra.run.smithi107.stdout:4285 punch from 0x24bd600 to 0x24bee00, (0x1800 bytes) 2024-09-11T09:30:00.367 INFO:teuthology.orchestra.run.smithi107.stdout:4286 read 0x3a3f000 thru 0x3a4d10d (0xe10e bytes) 2024-09-11T09:30:00.367 INFO:teuthology.orchestra.run.smithi107.stdout:4288 read 0x1c7b400 thru 0x1c7ea24 (0x3625 bytes) 2024-09-11T09:30:00.368 INFO:teuthology.orchestra.run.smithi107.stdout:4291 punch from 0xfdfc00 to 0xfe6400, (0x6800 bytes) 2024-09-11T09:30:00.371 INFO:teuthology.orchestra.run.smithi107.stdout:4293 trunc from 0x48f4800 to 0x202a000 2024-09-11T09:30:00.412 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:00.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:00.412 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:00.413 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:00.414 INFO:teuthology.orchestra.run.smithi107.stdout:4295 write 0x3216e00 thru 0x3219f4b (0x314c bytes) 2024-09-11T09:30:00.415 INFO:teuthology.orchestra.run.smithi107.stdout:4296 punch from 0x64a600 to 0x64a800, (0x200 bytes) 2024-09-11T09:30:00.415 INFO:teuthology.orchestra.run.smithi107.stdout:4297 read 0x1095200 thru 0x109e670 (0x9471 bytes) 2024-09-11T09:30:00.416 INFO:teuthology.orchestra.run.smithi107.stdout:4299 write 0x374e00 thru 0x37ed14 (0x9f15 bytes) 2024-09-11T09:30:00.417 INFO:teuthology.orchestra.run.smithi107.stdout:4300 punch from 0x3173a00 to 0x3176000, (0x2600 bytes) 2024-09-11T09:30:00.422 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:00.431 INFO:teuthology.orchestra.run.smithi107.stdout:4301 write 0x87cb400 thru 0x87cc7d0 (0x13d1 bytes) 2024-09-11T09:30:00.438 INFO:teuthology.orchestra.run.smithi107.stdout:4302 punch from 0x2a2e200 to 0x2a31a00, (0x3800 bytes) 2024-09-11T09:30:00.457 INFO:teuthology.orchestra.run.smithi107.stdout:4303 write 0xb052000 thru 0xb05f390 (0xd391 bytes) 2024-09-11T09:30:00.461 INFO:teuthology.orchestra.run.smithi107.stdout:4304 punch from 0x6a4ae00 to 0x6a59000, (0xe200 bytes) 2024-09-11T09:30:00.462 INFO:teuthology.orchestra.run.smithi107.stdout:4305 write 0x7656a00 thru 0x765ac9e (0x429f bytes) 2024-09-11T09:30:00.466 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:00.466+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:00.475 INFO:teuthology.orchestra.run.smithi107.stdout:4307 trunc from 0xb05f400 to 0x48d200 2024-09-11T09:30:00.516 INFO:teuthology.orchestra.run.smithi107.stdout:4310 punch from 0xa4600 to 0xa6000, (0x1a00 bytes) 2024-09-11T09:30:00.531 INFO:teuthology.orchestra.run.smithi107.stdout:4314 write 0x35d1600 thru 0x35d7142 (0x5b43 bytes) 2024-09-11T09:30:00.544 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:00.544 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:00.544 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:00.544 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:00.545 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:00.552 INFO:teuthology.orchestra.run.smithi107.stdout:4318 write 0x93daa00 thru 0x93e7eb2 (0xd4b3 bytes) 2024-09-11T09:30:00.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:00.555 INFO:teuthology.orchestra.run.smithi107.stdout:4319 read 0x8277200 thru 0x8277684 (0x485 bytes) 2024-09-11T09:30:00.555 INFO:teuthology.orchestra.run.smithi107.stdout:4320 write 0x103b400 thru 0x1043dbb (0x89bc bytes) 2024-09-11T09:30:00.556 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:00.555+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:00.569 INFO:teuthology.orchestra.run.smithi107.stdout:4321 write 0xa34c600 thru 0xa356629 (0xa02a bytes) 2024-09-11T09:30:00.570 INFO:teuthology.orchestra.run.smithi107.stdout:4323 read 0x95f9a00 thru 0x9607557 (0xdb58 bytes) 2024-09-11T09:30:00.570 INFO:teuthology.orchestra.run.smithi107.stdout:4324 punch from 0x2b49c00 to 0x2b4c400, (0x2800 bytes) 2024-09-11T09:30:00.575 INFO:teuthology.orchestra.run.smithi107.stdout:4326 punch from 0x8439e00 to 0x843b600, (0x1800 bytes) 2024-09-11T09:30:00.576 INFO:teuthology.orchestra.run.smithi107.stdout:4332 read 0x64dc00 thru 0x65a52b (0xc92c bytes) 2024-09-11T09:30:00.576 INFO:teuthology.orchestra.run.smithi107.stdout:4335 write 0x3eeae00 thru 0x3ef0594 (0x5795 bytes) 2024-09-11T09:30:00.576 INFO:teuthology.orchestra.run.smithi107.stdout:4336 punch from 0x4abfa00 to 0x4ac3200, (0x3800 bytes) 2024-09-11T09:30:00.577 INFO:teuthology.orchestra.run.smithi107.stdout:4338 read 0x46a9c00 thru 0x46ace92 (0x3293 bytes) 2024-09-11T09:30:00.577 INFO:teuthology.orchestra.run.smithi107.stdout:4339 write 0x6865000 thru 0x686d1a6 (0x81a7 bytes) 2024-09-11T09:30:00.577 INFO:teuthology.orchestra.run.smithi107.stdout:4341 write 0x3d38600 thru 0x3d3e541 (0x5f42 bytes) 2024-09-11T09:30:00.577 INFO:teuthology.orchestra.run.smithi107.stdout:4343 punch from 0x7248600 to 0x7256400, (0xde00 bytes) 2024-09-11T09:30:00.589 INFO:teuthology.orchestra.run.smithi107.stdout:4345 write 0xed58400 thru 0xed68134 (0xfd35 bytes) 2024-09-11T09:30:00.590 INFO:teuthology.orchestra.run.smithi107.stdout:4346 write 0x4513400 thru 0x451d2b7 (0x9eb8 bytes) 2024-09-11T09:30:00.590 INFO:teuthology.orchestra.run.smithi107.stdout:4347 write 0x6c19a00 thru 0x6c2470c (0xad0d bytes) 2024-09-11T09:30:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:4348 trunc from 0xed68200 to 0xc6f7a00 2024-09-11T09:30:00.614 INFO:teuthology.orchestra.run.smithi107.stdout:4350 write 0x50fcc00 thru 0x50fceb5 (0x2b6 bytes) 2024-09-11T09:30:00.614 INFO:teuthology.orchestra.run.smithi107.stdout:4352 write 0x7069e00 thru 0x7078008 (0xe209 bytes) 2024-09-11T09:30:00.616 INFO:teuthology.orchestra.run.smithi107.stdout:4353 trunc from 0xc6f7a00 to 0xaaa2000 2024-09-11T09:30:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:4354 read 0x179d800 thru 0x17a7989 (0xa18a bytes) 2024-09-11T09:30:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:4356 read 0x9fc400 thru 0xa05f92 (0x9b93 bytes) 2024-09-11T09:30:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:4360 read 0x1f8600 thru 0x2064af (0xdeb0 bytes) 2024-09-11T09:30:00.650 INFO:teuthology.orchestra.run.smithi107.stdout:4361 write 0xd6d7c00 thru 0xd6dc9d8 (0x4dd9 bytes) 2024-09-11T09:30:00.651 INFO:teuthology.orchestra.run.smithi107.stdout:4362 read 0x789bc00 thru 0x78a3216 (0x7617 bytes) 2024-09-11T09:30:00.651 INFO:teuthology.orchestra.run.smithi107.stdout:4364 punch from 0x5402800 to 0x5403e00, (0x1600 bytes) 2024-09-11T09:30:00.655 INFO:teuthology.orchestra.run.smithi107.stdout:4367 punch from 0x5e4fe00 to 0x5e57200, (0x7400 bytes) 2024-09-11T09:30:00.655 INFO:teuthology.orchestra.run.smithi107.stdout:4368 punch from 0x4ca6000 to 0x4cb3200, (0xd200 bytes) 2024-09-11T09:30:00.656 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:00.656+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:00.665 INFO:teuthology.orchestra.run.smithi107.stdout:4369 trunc from 0xd6dca00 to 0x4f91400 2024-09-11T09:30:00.696 INFO:teuthology.orchestra.run.smithi107.stdout:4370 write 0xb64600 thru 0xb67f51 (0x3952 bytes) 2024-09-11T09:30:00.698 INFO:teuthology.orchestra.run.smithi107.stdout:4371 read 0x4c6f000 thru 0x4c7acb7 (0xbcb8 bytes) 2024-09-11T09:30:00.698 INFO:teuthology.orchestra.run.smithi107.stdout:4372 read 0x475d400 thru 0x4768d51 (0xb952 bytes) 2024-09-11T09:30:00.710 INFO:teuthology.orchestra.run.smithi107.stdout:4373 write 0x6035400 thru 0x603c789 (0x738a bytes) 2024-09-11T09:30:00.711 INFO:teuthology.orchestra.run.smithi107.stdout:4374 read 0x3bb5e00 thru 0x3bbdd1d (0x7f1e bytes) 2024-09-11T09:30:00.711 INFO:teuthology.orchestra.run.smithi107.stdout:4376 punch from 0x324d000 to 0x3254000, (0x7000 bytes) 2024-09-11T09:30:00.720 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.737216682000003, 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:00.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.737170638000002, 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.726 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.737158968000003, 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:00.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735870699000003, 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:00.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735834998000001, 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.741 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:00.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.745 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735812961000001, 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:00.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735777376000001, 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.750 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:00.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.754 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735717033, 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:00.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735668349000001, 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:00.759 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:00.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:00.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735611884999997, 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.764 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:00.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735592330000003, 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:00.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.773 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735544566999998, 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735430829000002, 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.778 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:00.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:00.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735412085, 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:00.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735373096000004, 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.735349988999999, 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 31.027273498, 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.664874168000001, 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:00.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:00.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.659623091, 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:00.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:00.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.812 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 13.659362721000001, 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:00.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:00.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:00.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:00.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:00.819 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:00.819 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:30:00.821 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:00.820+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:4379 write 0xca3c000 thru 0xca43117 (0x7118 bytes) 2024-09-11T09:30:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:4382 write 0xa48ce00 thru 0xa498870 (0xba71 bytes) 2024-09-11T09:30:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:4383 punch from 0x3ef6000 to 0x3f05c00, (0xfc00 bytes) 2024-09-11T09:30:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:4384 trunc from 0xca43200 to 0x44bd200 2024-09-11T09:30:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:4385 write 0x2292400 thru 0x22a2287 (0xfe88 bytes) 2024-09-11T09:30:00.822 INFO:teuthology.orchestra.run.smithi107.stdout:4386 read 0x3fbfe00 thru 0x3fcdfee (0xe1ef bytes) 2024-09-11T09:30:00.823 INFO:teuthology.orchestra.run.smithi107.stdout:4390 clone 29 order 22 su 1048576 sc 8 2024-09-11T09:30:00.823 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:30:00.887 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone28 from 0x44bd200 (overlap 0x48d200) to 0x0 2024-09-11T09:30:00.921 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:00.921+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:00.947 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:00.947 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:00.947 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:00.947 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:00.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:30:01.022 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:01.022+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:01.107 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:01.108 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:01.116 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:30:01.122 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:01.123+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:01.223 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:01.223+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:01.229 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:01.229 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1767 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 172032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 55.352157877000003, 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024840193999999999, 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.230 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1767 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:30:01.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876702+0000", 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9099999999999999e-07 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876708+0000", 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7960000000000001e-06 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876716+0000", 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7000000000000008e-06 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876721+0000", 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901424+0000", 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024702923000000002 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901476+0000", 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1653000000000002e-05 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901542+0000", 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6492e-05 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 55.352131751000002, 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:30:01.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 55.352109466000002, 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.239 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:30:01.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 55.352086026000002, 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.244 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:01.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 55.352066178999998, 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.248 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:30:01.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:30:01.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.252 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 55.352033802000001, 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:01.253 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:30:01.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:548 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:30:01.257 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.202539829999999, 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026786889000000001, 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 548 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026321+0000", 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8599999999999996e-07 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026326+0000", 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2669999999999997e-06 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026342+0000", 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5695000000000001e-05 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026350+0000", 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:30:01.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026418+0000", 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7268999999999996e-05 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026621+0000", 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000203172 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053107+0000", 2024-09-11T09:30:01.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026486129000000001 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:558 3.1f 3:fa19977a:::rbd_data.1106b643ff7f.000000000000001a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.202397264, 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026276384, 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.262 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 558 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1200000000000002e-07 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026468+0000", 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2729999999999996e-06 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026508+0000", 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0095000000000002e-05 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026516+0000", 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6549999999999993e-06 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026689+0000", 2024-09-11T09:30:01.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017269599999999999 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026791+0000", 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010203 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.052739+0000", 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025947923000000001 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:565 3.1f 3:fa508b4a:::rbd_data.1106b643ff7f.000000000000001e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.202319982000001, 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026702457999999998, 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 565 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.267 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026541+0000", 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9999999999999997e-07 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026546+0000", 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6829999999999996e-06 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026554+0000", 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6869999999999994e-06 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026559+0000", 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027294+0000", 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000734361 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027468+0000", 2024-09-11T09:30:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017458400000000001 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053242+0000", 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025774047000000001 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:595 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.271 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.201909835, 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026285078, 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 595 2024-09-11T09:30:01.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026951+0000", 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3799999999999997e-07 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026954+0000", 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0599999999999999e-06 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026961+0000", 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3189999999999997e-06 2024-09-11T09:30:01.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026967+0000", 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5890000000000002e-06 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027011+0000", 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4682999999999999e-05 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027107+0000", 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5419000000000002e-05 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053235+0000", 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026128470000000001 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.276 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:629 3.1f 3:fb0b56ac:::rbd_data.1106b643ff7f.0000000000000048:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.201476575999999, 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02599951, 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 629 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.277 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027384+0000", 2024-09-11T09:30:01.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.23e-07 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027391+0000", 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0899999999999999e-06 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027408+0000", 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.717e-05 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027416+0000", 2024-09-11T09:30:01.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1570000000000003e-06 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027791+0000", 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00037519499999999999 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027902+0000", 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111215 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053383+0000", 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025480860000000001 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:638 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.201371982, 2024-09-11T09:30:01.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025900833000000002, 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 638 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027489+0000", 2024-09-11T09:30:01.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0299999999999998e-07 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027492+0000", 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6270000000000002e-06 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027501+0000", 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8440000000000004e-06 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027507+0000", 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0689999999999998e-06 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028168+0000", 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066121099999999998 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028276+0000", 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107435 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053389+0000", 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025112944000000002 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:660 3.12 3:4f6d0c71:::rbd_data.1106b643ff7f.000000000000005c:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.200714795, 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025370475, 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.287 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 660 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028146+0000", 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2900000000000004e-07 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028149+0000", 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9000000000000002e-06 2024-09-11T09:30:01.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028156+0000", 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2200000000000003e-06 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028161+0000", 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6910000000000002e-06 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028580+0000", 2024-09-11T09:30:01.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000418625 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028685+0000", 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010467300000000001 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053516+0000", 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024830837000000001 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1000 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.173877893, 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037281880000000003, 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1000 2024-09-11T09:30:01.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054983+0000", 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054987+0000", 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1670000000000001e-06 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054996+0000", 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7490000000000004e-06 2024-09-11T09:30:01.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055002+0000", 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4799999999999998e-06 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055397+0000", 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00039521700000000001 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055474+0000", 2024-09-11T09:30:01.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6872000000000005e-05 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055665+0000", 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000191013 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056748+0000", 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010830779999999999 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:01.296 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092163+0000", 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035415042000000001 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092225+0000", 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1555000000000003e-05 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092264+0000", 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9020999999999998e-05 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1010 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.173720941999999, 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037320416000000002, 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.298 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1010 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2199999999999999e-07 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055142+0000", 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5620000000000002e-06 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055148+0000", 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2400000000000004e-06 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055153+0000", 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.977e-06 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056168+0000", 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010149040000000001 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056237+0000", 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8606e-05 2024-09-11T09:30:01.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056413+0000", 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017639199999999999 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057731+0000", 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001317616 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092370+0000", 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034638872000000001 2024-09-11T09:30:01.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092419+0000", 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8989999999999997e-05 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092459+0000", 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0834999999999998e-05 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1018 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.171872484, 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035657162999999999, 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1018 2024-09-11T09:30:01.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056988+0000", 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.95e-07 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056994+0000", 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4500000000000003e-06 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057005+0000", 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1522000000000001e-05 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057013+0000", 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6599999999999995e-06 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057058+0000", 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057153+0000", 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5377000000000005e-05 2024-09-11T09:30:01.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057338+0000", 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018502199999999999 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057925+0000", 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00058726299999999998 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:01.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092552+0000", 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034627003000000003 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092598+0000", 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5989000000000003e-05 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092645+0000", 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6431999999999999e-05 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.309 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1086 3.12 3:4b95dab8:::rbd_data.1106b643ff7f.0000000000000085:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 14.168218979000001, 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.032140305000000001, 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1086 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.310 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060642+0000", 2024-09-11T09:30:01.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4300000000000002e-07 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060650+0000", 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1969999999999996e-06 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060660+0000", 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0471e-05 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060667+0000", 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4829999999999996e-06 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060798+0000", 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013138599999999999 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060884+0000", 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5348999999999999e-05 2024-09-11T09:30:01.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061065+0000", 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018090700000000001 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061845+0000", 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00078057400000000002 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092700+0000", 2024-09-11T09:30:01.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.030854940000000001 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092749+0000", 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9101999999999998e-05 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092781+0000", 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2053e-05 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5323.0:832 3.18 3:18760478:::rbd_header.1106a471d92f:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4629222259999999, 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3939045349999999, 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5323", 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2665032934", 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 832 2024-09-11T09:30:01.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.317 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765940+0000", 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765952+0000", 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1708999999999999e-05 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765970+0000", 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8459999999999999e-05 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765984+0000", 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4056999999999999e-05 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766066+0000", 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1976000000000003e-05 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766075+0000", 2024-09-11T09:30:01.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2940000000000006e-06 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768269+0000", 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021941510000000001 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768283+0000", 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3966e-05 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.320 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.157333+0000", 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3890506730000001 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.157618+0000", 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000284553 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 2", 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158170+0000", 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00055215699999999995 2024-09-11T09:30:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159321+0000", 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011509669999999999 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159683+0000", 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00036161199999999998 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159755+0000", 2024-09-11T09:30:01.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1730999999999995e-05 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159842+0000", 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7749000000000003e-05 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5323.0:833 3.18 3:18760478:::rbd_header.1106a471d92f:head [watch reconnect cookie 140016404064176 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:30:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4628430179999992, 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.395538188, 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5323", 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2665032934", 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 833 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:30:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766018+0000", 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6199999999999996e-07 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766023+0000", 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8239999999999999e-06 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766034+0000", 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1231e-05 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766042+0000", 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6720000000000004e-06 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766151+0000", 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109078 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766156+0000", 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6729999999999997e-06 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768984+0000", 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0028281130000000002 2024-09-11T09:30:01.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768994+0000", 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0012e-05 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158796+0000", 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.389801721 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158907+0000", 2024-09-11T09:30:01.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000110986 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159892+0000", 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00098477999999999999 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159989+0000", 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7326999999999998e-05 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.161555+0000", 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0015659090000000001 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:16052 3.18 3:18760478:::rbd_header.1106a471d92f:head [watch reconnect cookie 140373288956208 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4628054489999993, 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3957736299999999, 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.330 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 16052 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766056+0000", 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7460000000000001e-06 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766063+0000", 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7499999999999997e-06 2024-09-11T09:30:01.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766078+0000", 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4866e-05 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766087+0000", 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4180000000000003e-06 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.333 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766271+0000", 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018336900000000001 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766278+0000", 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9009999999999997e-06 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.769020+0000", 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0027424229999999999 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.769028+0000", 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7190000000000001e-06 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159133+0000", 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3901048120000001 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159234+0000", 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101895 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160503+0000", 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001268868 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160624+0000", 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000120931 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.161828+0000", 2024-09-11T09:30:01.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001203932 2024-09-11T09:30:01.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.337 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:01.338 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-11T09:30:01.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:01.339 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:01.338+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:01.439 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:01.439+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:01.467 INFO:teuthology.orchestra.run.smithi107.stdout:4392 write 0x489ce00 thru 0x489d95c (0xb5d bytes) 2024-09-11T09:30:01.468 INFO:teuthology.orchestra.run.smithi107.stdout:4393 read 0x730c00 thru 0x73d7e1 (0xcbe2 bytes) 2024-09-11T09:30:01.468 INFO:teuthology.orchestra.run.smithi107.stdout:4394 write 0x44cf400 thru 0x44d1c1a (0x281b bytes) 2024-09-11T09:30:01.481 INFO:teuthology.orchestra.run.smithi107.stdout:4395 write 0x52c8c00 thru 0x52c940f (0x810 bytes) 2024-09-11T09:30:01.490 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:01.492 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:01.492 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:01.492 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:01.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:01.504 INFO:teuthology.orchestra.run.smithi107.stdout:4396 write 0xd581000 thru 0xd585bbc (0x4bbd bytes) 2024-09-11T09:30:01.505 INFO:teuthology.orchestra.run.smithi107.stdout:4397 read 0x4f59e00 thru 0x4f63b03 (0x9d04 bytes) 2024-09-11T09:30:01.505 INFO:teuthology.orchestra.run.smithi107.stdout:4398 write 0xbebd800 thru 0xbec322f (0x5a30 bytes) 2024-09-11T09:30:01.505 INFO:teuthology.orchestra.run.smithi107.stdout:4400 write 0x94e0600 thru 0x94e8d2b (0x872c bytes) 2024-09-11T09:30:01.505 INFO:teuthology.orchestra.run.smithi107.stdout:4401 read 0x490c200 thru 0x491319a (0x6f9b bytes) 2024-09-11T09:30:01.505 INFO:teuthology.orchestra.run.smithi107.stdout:4405 read 0x725e00 thru 0x72b669 (0x586a bytes) 2024-09-11T09:30:01.506 INFO:teuthology.orchestra.run.smithi107.stdout:4406 punch from 0x742bc00 to 0x7435800, (0x9c00 bytes) 2024-09-11T09:30:01.509 INFO:teuthology.orchestra.run.smithi107.stdout:4407 write 0xc181600 thru 0xc18723a (0x5c3b bytes) 2024-09-11T09:30:01.509 INFO:teuthology.orchestra.run.smithi107.stdout:4408 read 0xa70f000 thru 0xa71917e (0xa17f bytes) 2024-09-11T09:30:01.509 INFO:teuthology.orchestra.run.smithi107.stdout:4409 write 0x27b4800 thru 0x27c11d5 (0xc9d6 bytes) 2024-09-11T09:30:01.516 INFO:teuthology.orchestra.run.smithi107.stdout:4410 trunc from 0xd585c00 to 0x7a7e600 2024-09-11T09:30:01.540 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:01.541+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:01.541 INFO:teuthology.orchestra.run.smithi107.stdout:4411 write 0x1e11e00 thru 0x1e164aa (0x46ab bytes) 2024-09-11T09:30:01.543 INFO:teuthology.orchestra.run.smithi107.stdout:4413 read 0x3e7e200 thru 0x3e88e12 (0xac13 bytes) 2024-09-11T09:30:01.543 INFO:teuthology.orchestra.run.smithi107.stdout:4415 read 0x5b66600 thru 0x5b6e910 (0x8311 bytes) 2024-09-11T09:30:01.543 INFO:teuthology.orchestra.run.smithi107.stdout:4417 punch from 0x1ba0200 to 0x1ba5000, (0x4e00 bytes) 2024-09-11T09:30:01.548 INFO:teuthology.orchestra.run.smithi107.stdout:4419 trunc from 0x7a7e600 to 0xa552400 2024-09-11T09:30:01.567 INFO:teuthology.orchestra.run.smithi107.stdout:4421 punch from 0x5b2f600 to 0x5b3c000, (0xca00 bytes) 2024-09-11T09:30:01.569 INFO:teuthology.orchestra.run.smithi107.stdout:4422 write 0x2f8400 thru 0x30820d (0xfe0e bytes) 2024-09-11T09:30:01.592 INFO:teuthology.orchestra.run.smithi107.stdout:4423 write 0xaf30000 thru 0xaf34485 (0x4486 bytes) 2024-09-11T09:30:01.593 INFO:teuthology.orchestra.run.smithi107.stdout:4426 read 0x48c6200 thru 0x48d42ee (0xe0ef bytes) 2024-09-11T09:30:01.593 INFO:teuthology.orchestra.run.smithi107.stdout:4427 write 0x7dc1a00 thru 0x7dce964 (0xcf65 bytes) 2024-09-11T09:30:01.593 INFO:teuthology.orchestra.run.smithi107.stdout:4429 read 0x758e00 thru 0x765038 (0xc239 bytes) 2024-09-11T09:30:01.594 INFO:teuthology.orchestra.run.smithi107.stdout:4430 write 0x6505600 thru 0x651061f (0xb020 bytes) 2024-09-11T09:30:01.594 INFO:teuthology.orchestra.run.smithi107.stdout:4431 punch from 0xdeec00 to 0xdfea00, (0xfe00 bytes) 2024-09-11T09:30:01.594 INFO:teuthology.orchestra.run.smithi107.stdout:4435 write 0xa3ad200 thru 0xa3b312b (0x5f2c bytes) 2024-09-11T09:30:01.607 INFO:teuthology.orchestra.run.smithi107.stdout:4437 trunc from 0xaf34600 to 0x50a600 2024-09-11T09:30:01.641 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:01.642+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:01.659 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:01.659 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:01.659 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:01.659 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:01.659 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:01.660 INFO:teuthology.orchestra.run.smithi107.stdout:4438 trunc from 0x50a600 to 0xeaf5000 2024-09-11T09:30:01.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:01.671 INFO:teuthology.orchestra.run.smithi107.stdout:4439 flatten 2024-09-11T09:30:01.684 INFO:teuthology.orchestra.run.smithi107.stdout:4440 read 0x475ee00 thru 0x476a6b7 (0xb8b8 bytes) 2024-09-11T09:30:01.684 INFO:teuthology.orchestra.run.smithi107.stdout:4441 write 0x513e600 thru 0x5141cb3 (0x36b4 bytes) 2024-09-11T09:30:01.692 INFO:teuthology.orchestra.run.smithi107.stdout:4444 trunc from 0xeaf5000 to 0x8174800 2024-09-11T09:30:01.706 INFO:teuthology.orchestra.run.smithi107.stdout:4445 read 0x9c7400 thru 0x9d448b (0xd08c bytes) 2024-09-11T09:30:01.707 INFO:teuthology.orchestra.run.smithi107.stdout:4446 punch from 0x1bbce00 to 0x1bc2800, (0x5a00 bytes) 2024-09-11T09:30:01.721 INFO:teuthology.orchestra.run.smithi107.stdout:4447 write 0xaed7800 thru 0xaed8a68 (0x1269 bytes) 2024-09-11T09:30:01.733 INFO:teuthology.orchestra.run.smithi107.stdout:4448 trunc from 0xaed8c00 to 0xce9a00 2024-09-11T09:30:01.742 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:01.742+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:01.776 INFO:teuthology.orchestra.run.smithi107.stdout:4450 write 0xd43d400 thru 0xd4433d2 (0x5fd3 bytes) 2024-09-11T09:30:01.783 INFO:teuthology.orchestra.run.smithi107.stdout:4451 read 0x26d8400 thru 0x26dc145 (0x3d46 bytes) 2024-09-11T09:30:01.783 INFO:teuthology.orchestra.run.smithi107.stdout:4452 read 0x4b2c600 thru 0x4b37e47 (0xb848 bytes) 2024-09-11T09:30:01.783 INFO:teuthology.orchestra.run.smithi107.stdout:4455 write 0x7105e00 thru 0x7108951 (0x2b52 bytes) 2024-09-11T09:30:01.783 INFO:teuthology.orchestra.run.smithi107.stdout:4458 write 0x16a6a00 thru 0x16a7040 (0x641 bytes) 2024-09-11T09:30:01.786 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:01.786 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.875471392999998, 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.787 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.791 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.875240820999998, 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.792 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:01.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:01.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.796 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.875143616999999, 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.797 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:01.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:01.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.874908853000001, 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.802 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:01.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.874712836999997, 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.807 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:01.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.811 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.874444400000002, 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.812 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:01.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.816 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.874342650000003, 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:01.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:01.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.821 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.873948992999999, 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:01.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:01.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.873822748000002, 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.826 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:01.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.873613366000001, 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.831 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:01.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.835 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.873502416999997, 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:01.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.873246479000002, 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:01.840 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:01.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.873093949999998, 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:01.845 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:01.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.872901585999998, 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:01.850 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:01.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:01.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.872571168, 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:01.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:01.859 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.872294838000002, 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.860 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:01.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.872240918000003, 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.865 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:01.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.872040349000002, 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.870 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:01.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.873 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:01.874 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.871684809999998, 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:01.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 19.092419277000001, 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:01.880 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:01.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:01.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:01.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:01.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:01.887 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:01.887 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:30:01.887 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:01.887 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-11T09:30:01.887 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T09:30:01.887 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T09:30:01.888 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-11T09:30:01.890 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:01.889+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:01.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:01.892 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:30:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:4459 trunc from 0xd443400 to 0x1a5a200 2024-09-11T09:30:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:4461 write 0xaf20400 thru 0xaf2b021 (0xac22 bytes) 2024-09-11T09:30:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:4463 read 0x9051600 thru 0x905dd5d (0xc75e bytes) 2024-09-11T09:30:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:4465 write 0x5825a00 thru 0x582cff5 (0x75f6 bytes) 2024-09-11T09:30:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:4467 write 0xa54e000 thru 0xa55c62e (0xe62f bytes) 2024-09-11T09:30:01.893 INFO:teuthology.orchestra.run.smithi107.stdout:4471 write 0x8cf1400 thru 0x8cf1ec8 (0xac9 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4472 punch from 0x61a9800 to 0x61b4a00, (0xb200 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4473 read 0x2274a00 thru 0x2280728 (0xbd29 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4474 read 0x8ee3a00 thru 0x8eef961 (0xbf62 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4476 write 0xda1a200 thru 0xda29469 (0xf26a bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4477 trunc from 0xda29600 to 0xdb17000 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4478 read 0x8a61a00 thru 0x8a65bc3 (0x41c4 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4482 write 0x9dede00 thru 0x9dfa763 (0xc964 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4484 punch from 0x920bc00 to 0x9217e00, (0xc200 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4485 write 0xaac0000 thru 0xaac9f74 (0x9f75 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4488 punch from 0x6c99200 to 0x6c9ce00, (0x3c00 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4489 punch from 0x7052400 to 0x7052600, (0x200 bytes) 2024-09-11T09:30:01.894 INFO:teuthology.orchestra.run.smithi107.stdout:4494 punch from 0xda19c00 to 0xda24200, (0xa600 bytes) 2024-09-11T09:30:01.895 INFO:teuthology.orchestra.run.smithi107.stdout:4495 trunc from 0xdb17000 to 0x7a1dc00 2024-09-11T09:30:01.912 INFO:teuthology.orchestra.run.smithi107.stdout:4496 punch from 0x426400 to 0x435000, (0xec00 bytes) 2024-09-11T09:30:01.919 INFO:teuthology.orchestra.run.smithi107.stdout:4500 read 0x3fca800 thru 0x3fcfbbe (0x53bf bytes) 2024-09-11T09:30:01.919 INFO:teuthology.orchestra.run.smithi107.stdout:4501 read 0x61dae00 thru 0x61e6805 (0xba06 bytes) 2024-09-11T09:30:01.919 INFO:teuthology.orchestra.run.smithi107.stdout:4502 write 0x2eef600 thru 0x2ef7e86 (0x8887 bytes) 2024-09-11T09:30:01.919 INFO:teuthology.orchestra.run.smithi107.stdout:4503 write 0x46b9a00 thru 0x46bf563 (0x5b64 bytes) 2024-09-11T09:30:01.919 INFO:teuthology.orchestra.run.smithi107.stdout:4504 punch from 0x6ea0c00 to 0x6eac000, (0xb400 bytes) 2024-09-11T09:30:01.919 INFO:teuthology.orchestra.run.smithi107.stdout:4507 read 0x48a7e00 thru 0x48b10d8 (0x92d9 bytes) 2024-09-11T09:30:01.922 INFO:teuthology.orchestra.run.smithi107.stdout:4509 trunc from 0x7a1dc00 to 0x49d4a00 2024-09-11T09:30:01.967 INFO:teuthology.orchestra.run.smithi107.stdout:4510 write 0x7d05600 thru 0x7d0dee7 (0x88e8 bytes) 2024-09-11T09:30:01.980 INFO:teuthology.orchestra.run.smithi107.stdout:4512 write 0x9103800 thru 0x91079f4 (0x41f5 bytes) 2024-09-11T09:30:01.981 INFO:teuthology.orchestra.run.smithi107.stdout:4513 clone 30 order 21 su 16384 sc 11 2024-09-11T09:30:01.990 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:01.991+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:02.003 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:02.003 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "" 2024-09-11T09:30:02.003 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:02.044 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:02.044 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:02.044 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:02.044 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:02.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:02.053 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:02.096 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:02.096+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:02.162 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:02.162 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:02.162 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:02.162 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:02.162 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:02.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:02.177 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:30:02.177 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:30:02.194 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:02.195 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:02.195 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:02.194+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:02.211 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:02.245 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:02.245 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:02.277 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:02.277 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:02.293 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:02.293+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:02.299 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:02.300 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:02.311 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:02.311 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:02.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:02.394 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:02.394+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:02.424 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:02.424 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:02.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:02.494 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:02.494+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:02.574 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:02.574 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:02.583 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:02.627 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:02.627+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:02.686 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:02.686 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:02.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:02.697 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:02.697+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:02.797 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:02.798+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:02.841 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:02.841 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:02.851 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:02.898 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:02.897+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:02.996 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:02.996 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:02.999 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:02.999+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:03.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:03.100 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:03.100+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:03.149 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:03.150 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:03.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:03.201 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:03.201+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-11T09:30:03.301 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:03.301+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:03.303 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:03.303 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:03.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:03.336 INFO:teuthology.orchestra.run.smithi107.stdout:4515 trunc from 0x9107a00 to 0x8811400 2024-09-11T09:30:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:4518 read 0x59de600 thru 0x59e3251 (0x4c52 bytes) 2024-09-11T09:30:03.372 INFO:teuthology.orchestra.run.smithi107.stdout:4519 punch from 0x23c0c00 to 0x23cfc00, (0xf000 bytes) 2024-09-11T09:30:03.382 INFO:teuthology.orchestra.run.smithi107.stdout:4521 trunc from 0x8811400 to 0x1eb9200 2024-09-11T09:30:03.402 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:03.402+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:03.445 INFO:teuthology.orchestra.run.smithi107.stdout:4522 write 0x789ac00 thru 0x78a74ce (0xc8cf bytes) 2024-09-11T09:30:03.463 INFO:teuthology.orchestra.run.smithi107.stdout:4523 write 0x9398c00 thru 0x939dd36 (0x5137 bytes) 2024-09-11T09:30:03.470 INFO:teuthology.orchestra.run.smithi107.stdout:4524 write 0x5776600 thru 0x578149c (0xae9d bytes) 2024-09-11T09:30:03.483 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:03.483 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:03.487 INFO:teuthology.orchestra.run.smithi107.stdout:4525 write 0xa7bce00 thru 0xa7c7d14 (0xaf15 bytes) 2024-09-11T09:30:03.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:03.498 INFO:teuthology.orchestra.run.smithi107.stdout:4526 trunc from 0xa7c7e00 to 0x6e79c00 2024-09-11T09:30:03.526 INFO:teuthology.orchestra.run.smithi107.stdout:4528 write 0xad0600 thru 0xad9d24 (0x9725 bytes) 2024-09-11T09:30:03.533 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:03.534+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:03.535 INFO:teuthology.orchestra.run.smithi107.stdout:4534 trunc from 0x6e79c00 to 0xe462c00 2024-09-11T09:30:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:4535 read 0xa638000 thru 0xa644c14 (0xcc15 bytes) 2024-09-11T09:30:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:4536 write 0xcdea00 thru 0xce4cac (0x62ad bytes) 2024-09-11T09:30:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:4537 punch from 0x6b43400 to 0x6b44a00, (0x1600 bytes) 2024-09-11T09:30:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:4538 write 0xa251200 thru 0xa2596d4 (0x84d5 bytes) 2024-09-11T09:30:03.581 INFO:teuthology.orchestra.run.smithi107.stdout:4539 trunc from 0xe462c00 to 0xb1a4800 2024-09-11T09:30:03.604 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:03.604+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:03.608 INFO:teuthology.orchestra.run.smithi107.stdout:4542 clone 31 order 25 su 524288 sc 11 2024-09-11T09:30:03.651 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:03.651 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:03.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:30:03.705 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:03.705+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:03.768 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:03.768 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:03.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:30:03.820 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:03.820+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:03.877 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone30 from 0xb1a4800 (overlap 0x1eb9200) to 0x1894000 2024-09-11T09:30:03.890 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:03.890 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:03.894 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:30:03.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:30:03.905 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:03.906+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:04.007 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:04.007+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:04.052 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:04.052 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:04.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:04.108 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:04.107+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:04.173 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:04.173 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:04.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:04.194 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-11T09:30:04.209 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:04.209+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:04.310 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:04.309+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:04.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:04.338 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:04.339 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:04.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:04.433 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:04.433+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:04.443 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:30:04.443 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:30:04.462 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:04.462 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:04.479 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:04.479 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:04.497 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:04.497 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:04.504 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:04.504 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:04.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:04.504 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:04.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.592789427, 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.505 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:04.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:04.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:04.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.592558855, 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.510 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:04.511 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:04.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.592461651000001, 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:04.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.516 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:04.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.519 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.592226887000002, 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:04.521 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.592030870999999, 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:04.524 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:04.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.591762434000003, 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.529 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:04.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:04.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.591660683999997, 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.534 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:04.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:04.538 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.591267027000001, 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:04.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.543 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.591140781999997, 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:04.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:04.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.590931400000002, 2024-09-11T09:30:04.548 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:04.549 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:04.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.590820450999999, 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:04.553 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:04.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.556 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:04.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.590564512999997, 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.558 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:04.559 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:04.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.590411983999999, 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.563 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.564 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:04.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:04.567 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.590219619999999, 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:04.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.589889202000002, 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.572 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:04.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.575 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:04.576 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.589612872000004, 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.577 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.579 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.589558951999997, 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:04.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.583 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:04.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.586 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.589358382999997, 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.587 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:04.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:04.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.589002843999999, 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.591 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:04.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.595 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.809737310999999, 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:04.596 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:04.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:04.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:04.602 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.603 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.603 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.603 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:04.603 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:04.603 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:04.604 INFO:teuthology.orchestra.run.smithi107.stdout:4543 write 0xa0aca00 thru 0xa0b4e66 (0x8467 bytes) 2024-09-11T09:30:04.605 INFO:teuthology.orchestra.run.smithi107.stdout:4546 write 0x6f99a00 thru 0x6fa8517 (0xeb18 bytes) 2024-09-11T09:30:04.605 INFO:teuthology.orchestra.run.smithi107.stdout:4548 read 0x17b7a00 thru 0x17bd414 (0x5a15 bytes) 2024-09-11T09:30:04.605 INFO:teuthology.orchestra.run.smithi107.stdout:4550 punch from 0x2ac8000 to 0x2acc000, (0x4000 bytes) 2024-09-11T09:30:04.605 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:04.606 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:04.606 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:04.606 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:04.607 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:04.607 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:04.603+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:04.652 INFO:teuthology.orchestra.run.smithi107.stdout:4551 read 0x36d6400 thru 0x36e4572 (0xe173 bytes) 2024-09-11T09:30:04.652 INFO:teuthology.orchestra.run.smithi107.stdout:4552 write 0x2830e00 thru 0x283a852 (0x9a53 bytes) 2024-09-11T09:30:04.657 INFO:teuthology.orchestra.run.smithi107.stdout:4557 trunc from 0xb1a4800 to 0xe85f200 2024-09-11T09:30:04.668 INFO:teuthology.orchestra.run.smithi107.stdout:4560 punch from 0x8822a00 to 0x8830000, (0xd600 bytes) 2024-09-11T09:30:04.675 INFO:teuthology.orchestra.run.smithi107.stdout:4561 read 0x5495000 thru 0x549bd0c (0x6d0d bytes) 2024-09-11T09:30:04.675 INFO:teuthology.orchestra.run.smithi107.stdout:4564 write 0x4258c00 thru 0x425e7e1 (0x5be2 bytes) 2024-09-11T09:30:04.678 INFO:teuthology.orchestra.run.smithi107.stdout:4565 trunc from 0xe85f200 to 0xc79a600 2024-09-11T09:30:04.705 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:04.704+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:04.779 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:04.779 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:04.779 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:04.779 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:04.787 INFO:teuthology.orchestra.run.smithi107.stdout:4566 read 0x45e5200 thru 0x45eaf37 (0x5d38 bytes) 2024-09-11T09:30:04.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:04.793 INFO:teuthology.orchestra.run.smithi107.stdout:4567 write 0xba72800 thru 0xba75bd3 (0x33d4 bytes) 2024-09-11T09:30:04.794 INFO:teuthology.orchestra.run.smithi107.stdout:4569 trunc from 0xc79a600 to 0xcbe5400 2024-09-11T09:30:04.805 INFO:teuthology.orchestra.run.smithi107.stdout:4570 write 0xc1c1e00 thru 0xc1cfafc (0xdcfd bytes) 2024-09-11T09:30:04.813 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:30:04.813 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:04.812+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:04.813 INFO:teuthology.orchestra.run.smithi107.stdout:4571 read 0x8583000 thru 0x858bb20 (0x8b21 bytes) 2024-09-11T09:30:04.814 INFO:teuthology.orchestra.run.smithi107.stdout:4572 read 0x2655600 thru 0x2661d09 (0xc70a bytes) 2024-09-11T09:30:04.816 INFO:teuthology.orchestra.run.smithi107.stdout:4574 punch from 0xfc9600 to 0xfcae00, (0x1800 bytes) 2024-09-11T09:30:04.816 INFO:teuthology.orchestra.run.smithi107.stdout:4575 write 0x3a1e000 thru 0x3a2b7f3 (0xd7f4 bytes) 2024-09-11T09:30:04.831 INFO:teuthology.orchestra.run.smithi107.stdout:4578 trunc from 0xcbe5400 to 0x34a400 2024-09-11T09:30:04.884 INFO:teuthology.orchestra.run.smithi107.stdout:4579 trunc from 0x34a400 to 0x4198200 2024-09-11T09:30:04.892 INFO:teuthology.orchestra.run.smithi107.stdout:4581 read 0x33d7400 thru 0x33d91ea (0x1deb bytes) 2024-09-11T09:30:04.906 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:04.906+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:04.912 INFO:teuthology.orchestra.run.smithi107.stdout:4582 write 0xc1aae00 thru 0xc1accdf (0x1ee0 bytes) 2024-09-11T09:30:04.926 INFO:teuthology.orchestra.run.smithi107.stdout:4584 write 0xe20fe00 thru 0xe216c97 (0x6e98 bytes) 2024-09-11T09:30:04.927 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:04.927 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:04.927 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:04.927 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:04.927 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:04.935 INFO:teuthology.orchestra.run.smithi107.stdout:4585 trunc from 0xe216e00 to 0xc2a9e00 2024-09-11T09:30:04.936 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:04.953 INFO:teuthology.orchestra.run.smithi107.stdout:4586 read 0x6215200 thru 0x621d214 (0x8015 bytes) 2024-09-11T09:30:04.954 INFO:teuthology.orchestra.run.smithi107.stdout:4587 write 0x88eda00 thru 0x88f40b1 (0x66b2 bytes) 2024-09-11T09:30:04.954 INFO:teuthology.orchestra.run.smithi107.stdout:4588 write 0x9472800 thru 0x947b469 (0x8c6a bytes) 2024-09-11T09:30:04.967 INFO:teuthology.orchestra.run.smithi107.stdout:4589 write 0xd4c3400 thru 0xd4c4d67 (0x1968 bytes) 2024-09-11T09:30:04.967 INFO:teuthology.orchestra.run.smithi107.stdout:4590 read 0xbd88200 thru 0xbd88d5e (0xb5f bytes) 2024-09-11T09:30:04.967 INFO:teuthology.orchestra.run.smithi107.stdout:4592 punch from 0x7e16000 to 0x7e25600, (0xf600 bytes) 2024-09-11T09:30:04.968 INFO:teuthology.orchestra.run.smithi107.stdout:4593 read 0xb29e600 thru 0xb2a3440 (0x4e41 bytes) 2024-09-11T09:30:04.971 INFO:teuthology.orchestra.run.smithi107.stdout:4594 trunc from 0xd4c4e00 to 0xae78600 2024-09-11T09:30:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:4595 write 0xa69a600 thru 0xa6a2cf5 (0x86f6 bytes) 2024-09-11T09:30:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:4596 punch from 0x1934200 to 0x1938800, (0x4600 bytes) 2024-09-11T09:30:04.989 INFO:teuthology.orchestra.run.smithi107.stdout:4599 punch from 0x3d89200 to 0x3d8f600, (0x6400 bytes) 2024-09-11T09:30:04.989 INFO:teuthology.orchestra.run.smithi107.stdout:4600 write 0x12d4c00 thru 0x12de52a (0x992b bytes) 2024-09-11T09:30:04.989 INFO:teuthology.orchestra.run.smithi107.stdout:4602 punch from 0x36f1000 to 0x36fbe00, (0xae00 bytes) 2024-09-11T09:30:04.990 INFO:teuthology.orchestra.run.smithi107.stdout:4603 read 0x9fc2e00 thru 0x9fcf0c6 (0xc2c7 bytes) 2024-09-11T09:30:04.990 INFO:teuthology.orchestra.run.smithi107.stdout:4604 read 0x106b600 thru 0x106bfff (0xa00 bytes) 2024-09-11T09:30:04.992 INFO:teuthology.orchestra.run.smithi107.stdout:4606 trunc from 0xae78600 to 0xb341e00 2024-09-11T09:30:05.001 INFO:teuthology.orchestra.run.smithi107.stdout:4608 write 0x684c000 thru 0x685b669 (0xf66a bytes) 2024-09-11T09:30:05.005 INFO:teuthology.orchestra.run.smithi107.stdout:4609 trunc from 0xb341e00 to 0xe3c7800 2024-09-11T09:30:05.016 INFO:teuthology.orchestra.run.smithi107.stdout:4610 punch from 0x5aab400 to 0x5ab1600, (0x6200 bytes) 2024-09-11T09:30:05.021 INFO:teuthology.orchestra.run.smithi107.stdout:4611 read 0x8c2e600 thru 0x8c34e64 (0x6865 bytes) 2024-09-11T09:30:05.021 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:05.021+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:05.023 INFO:teuthology.orchestra.run.smithi107.stdout:4614 punch from 0x94f4600 to 0x94fe800, (0xa200 bytes) 2024-09-11T09:30:05.036 INFO:teuthology.orchestra.run.smithi107.stdout:4615 trunc from 0xe3c7800 to 0x3e4ec00 2024-09-11T09:30:05.058 INFO:teuthology.orchestra.run.smithi107.stdout:4616 punch from 0x45ca00 to 0x466200, (0x9800 bytes) 2024-09-11T09:30:05.066 INFO:teuthology.orchestra.run.smithi107.stdout:4617 write 0x27df000 thru 0x27e132a (0x232b bytes) 2024-09-11T09:30:05.067 INFO:teuthology.orchestra.run.smithi107.stdout:4619 punch from 0x3511600 to 0x3517800, (0x6200 bytes) 2024-09-11T09:30:05.069 INFO:teuthology.orchestra.run.smithi107.stdout:4620 trunc from 0x3e4ec00 to 0x5945e00 2024-09-11T09:30:05.090 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:05.090 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1670 3.0 3:035efefb:::rbd_data.11064477c615.000000000000002b:21 [read 1851392~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.720983+0000", 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36945760599999999, 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016094623999999998, 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.091 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1670 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.720983+0000", 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.720983+0000", 2024-09-11T09:30:05.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.720983+0000", 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5200000000000002e-07 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.720985+0000", 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2079999999999999e-06 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.720991+0000", 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7869999999999998e-06 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.720997+0000", 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9059999999999999e-06 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.736898+0000", 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015901239000000001 2024-09-11T09:30:05.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.736971+0000", 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2569000000000003e-05 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.737077+0000", 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000106463 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1679 3.18 3:1ef2ea64:::rbd_data.11064477c615.000000000000002d:21 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.721070+0000", 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36936990800000002, 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016030984000000002, 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1679 2024-09-11T09:30:05.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.721070+0000", 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.721070+0000", 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.721071+0000", 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9999999999999998e-07 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.721073+0000", 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2740000000000002e-06 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.721081+0000", 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6499999999999996e-06 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.721086+0000", 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1089999999999997e-06 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.736944+0000", 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.015858311 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.737028+0000", 2024-09-11T09:30:05.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3731000000000002e-05 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.737101+0000", 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3509000000000004e-05 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1747 3.18 3:1a4f4d84:::rbd_data.11064477c615.000000000000000c:21 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722119+0000", 2024-09-11T09:30:05.100 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36832105199999998, 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016228218999999999, 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1747 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.101 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722119+0000", 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722119+0000", 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722120+0000", 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4499999999999997e-07 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.102 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722122+0000", 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0389999999999999e-06 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722128+0000", 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1890000000000004e-06 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722134+0000", 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1600000000000003e-06 2024-09-11T09:30:05.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738204+0000", 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016069645 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738278+0000", 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4412999999999994e-05 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738347+0000", 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9327999999999998e-05 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1788 3.8 3:1608b698:::rbd_data.11064477c615.0000000000000010:21 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722203+0000", 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36823703400000002, 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016180569999999998, 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.105 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1788 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722203+0000", 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722203+0000", 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722204+0000", 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.75e-07 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722206+0000", 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1299999999999999e-06 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722212+0000", 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.032e-06 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722218+0000", 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0730000000000001e-06 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738235+0000", 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016017470999999998 2024-09-11T09:30:05.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738315+0000", 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9419000000000007e-05 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738384+0000", 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8969999999999999e-05 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1826 3.0 3:06b143bf:::rbd_data.11064477c615.0000000000000015:21 [read 327680~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722225+0000", 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.368215709, 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016336074999999999, 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1826 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722225+0000", 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722225+0000", 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722225+0000", 2024-09-11T09:30:05.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.75e-07 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722227+0000", 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0849999999999999e-06 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722233+0000", 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1990000000000003e-06 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.112 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722238+0000", 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738374+0000", 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016135832999999999 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738455+0000", 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1039999999999997e-05 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.113 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738561+0000", 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105741 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1827 3.0 3:06b143bf:::rbd_data.11064477c615.0000000000000015:21 [read 851968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722244+0000", 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36819605799999999, 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016504850000000001, 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.114 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1827 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722244+0000", 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722244+0000", 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722245+0000", 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722247+0000", 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9819999999999998e-06 2024-09-11T09:30:05.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722254+0000", 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9589999999999998e-06 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722259+0000", 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0549999999999999e-06 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738617+0000", 2024-09-11T09:30:05.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016358115999999999 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738681+0000", 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3924000000000007e-05 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738749+0000", 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8461999999999996e-05 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1830 3.0 3:06b143bf:::rbd_data.11064477c615.0000000000000015:21 [read 1376256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722267+0000", 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36817355499999999, 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016643139000000001, 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1830 2024-09-11T09:30:05.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722267+0000", 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722267+0000", 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722267+0000", 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.32e-07 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722269+0000", 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1959999999999998e-06 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722276+0000", 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8120000000000002e-06 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722282+0000", 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5480000000000001e-06 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738785+0000", 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016503231 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738829+0000", 2024-09-11T09:30:05.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4041000000000001e-05 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738910+0000", 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0878999999999996e-05 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1832 3.0 3:06b143bf:::rbd_data.11064477c615.0000000000000015:21 [read 1900544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722288+0000", 2024-09-11T09:30:05.123 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36815221799999998, 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016826780999999999, 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1832 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722288+0000", 2024-09-11T09:30:05.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722288+0000", 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722288+0000", 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6899999999999998e-07 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722291+0000", 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1919999999999999e-06 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722297+0000", 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9070000000000004e-06 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722301+0000", 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.702e-06 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738954+0000", 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016653232 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739034+0000", 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9814999999999994e-05 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739115+0000", 2024-09-11T09:30:05.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0563999999999996e-05 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1840 3.18 3:19a9be0c:::rbd_data.11064477c615.0000000000000021:21 [read 344064~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722308+0000", 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36813244899999997, 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016824591, 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1840 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722308+0000", 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722308+0000", 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722308+0000", 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6300000000000001e-07 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722310+0000", 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1469999999999999e-06 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722316+0000", 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8629999999999996e-06 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722321+0000", 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3560000000000003e-06 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.738938+0000", 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016617690000000001 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739015+0000", 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7157999999999995e-05 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739132+0000", 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117014 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1842 3.18 3:19a9be0c:::rbd_data.11064477c615.0000000000000021:21 [read 868352~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722328+0000", 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.368112308, 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016949008000000002, 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1842 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722328+0000", 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722328+0000", 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722328+0000", 2024-09-11T09:30:05.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2399999999999999e-07 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722331+0000", 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2369999999999999e-06 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722339+0000", 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2579999999999995e-06 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722344+0000", 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1150000000000002e-06 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739163+0000", 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016818801000000001 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739212+0000", 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9530000000000002e-05 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739277+0000", 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4642999999999998e-05 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1844 3.18 3:19a9be0c:::rbd_data.11064477c615.0000000000000021:21 [read 1392640~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722351+0000", 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36808937899999999, 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017270276000000001, 2024-09-11T09:30:05.137 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1844 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722351+0000", 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722351+0000", 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722351+0000", 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722360+0000", 2024-09-11T09:30:05.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.473e-06 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722366+0000", 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5989999999999998e-06 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722372+0000", 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6200000000000004e-06 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739323+0000", 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.016950653999999999 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739473+0000", 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000150388 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739621+0000", 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014813200000000001 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1846 3.18 3:19a9be0c:::rbd_data.11064477c615.0000000000000021:21 [read 1916928~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722380+0000", 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36806013500000001, 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017503098000000002, 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1846 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722380+0000", 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722380+0000", 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722381+0000", 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6499999999999999e-07 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722383+0000", 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.469e-06 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722390+0000", 2024-09-11T09:30:05.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5969999999999997e-06 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722395+0000", 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1250000000000001e-06 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739693+0000", 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01729793 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739784+0000", 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1092000000000001e-05 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739883+0000", 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9320000000000005e-05 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.146 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1860 3.0 3:035efefb:::rbd_data.11064477c615.000000000000002b:21 [read 327680~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722401+0000", 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36803880100000003, 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017416442000000001, 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1860 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722401+0000", 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722401+0000", 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722402+0000", 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:30:05.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722404+0000", 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722412+0000", 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6769999999999995e-06 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722417+0000", 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7089999999999999e-06 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739656+0000", 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017238377999999999 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739739+0000", 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3078000000000007e-05 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739818+0000", 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9024999999999996e-05 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1861 3.0 3:035efefb:::rbd_data.11064477c615.000000000000002b:21 [read 851968~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722424+0000", 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36801600899999998, 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.01754118, 2024-09-11T09:30:05.151 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1861 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722424+0000", 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722424+0000", 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722425+0000", 2024-09-11T09:30:05.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8100000000000003e-07 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722427+0000", 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3120000000000001e-06 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722434+0000", 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7050000000000002e-06 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722439+0000", 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8899999999999998e-06 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739832+0000", 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017393676 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739883+0000", 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0964999999999997e-05 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.739965+0000", 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2150999999999993e-05 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1862 3.0 3:035efefb:::rbd_data.11064477c615.000000000000002b:21 [read 1376256~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722447+0000", 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36799369599999998, 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017733233000000001, 2024-09-11T09:30:05.156 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1862 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722447+0000", 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722447+0000", 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722447+0000", 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722449+0000", 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0439999999999998e-06 2024-09-11T09:30:05.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722455+0000", 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0909999999999998e-06 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722460+0000", 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3850000000000004e-06 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740004+0000", 2024-09-11T09:30:05.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017544566000000001 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740083+0000", 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8700000000000002e-05 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740180+0000", 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7057000000000003e-05 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1863 3.0 3:035efefb:::rbd_data.11064477c615.000000000000002b:21 [read 1900544~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722466+0000", 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36797407700000001, 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017956027999999999, 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1863 2024-09-11T09:30:05.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722466+0000", 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722466+0000", 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722467+0000", 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4999999999999998e-07 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722469+0000", 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0760000000000001e-06 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722475+0000", 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1210000000000003e-06 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722480+0000", 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9080000000000003e-06 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740257+0000", 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017777719000000001 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740329+0000", 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1863e-05 2024-09-11T09:30:05.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740422+0000", 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2891000000000002e-05 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1868 3.18 3:1ef2ea64:::rbd_data.11064477c615.000000000000002d:21 [read 344064~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722489+0000", 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36795142800000002, 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017889505999999999, 2024-09-11T09:30:05.165 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1868 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722489+0000", 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722489+0000", 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722489+0000", 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722491+0000", 2024-09-11T09:30:05.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.956e-06 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722497+0000", 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3730000000000001e-06 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722502+0000", 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2469999999999999e-06 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740206+0000", 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017704025000000002 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740286+0000", 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0489999999999997e-05 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740378+0000", 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2004999999999997e-05 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.169 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1869 3.18 3:1ef2ea64:::rbd_data.11064477c615.000000000000002d:21 [read 868352~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722509+0000", 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36793118899999999, 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.018056246000000001, 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.170 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1869 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722509+0000", 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722509+0000", 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722510+0000", 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2399999999999998e-07 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722512+0000", 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.3530000000000002e-06 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722518+0000", 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1820000000000003e-06 2024-09-11T09:30:05.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722523+0000", 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8439999999999997e-06 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740403+0000", 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.017879591 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740476+0000", 2024-09-11T09:30:05.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2946000000000003e-05 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740565+0000", 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.9806e-05 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1870 3.18 3:1ef2ea64:::rbd_data.11064477c615.000000000000002d:21 [read 1392640~49152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722530+0000", 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36791026599999999, 2024-09-11T09:30:05.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.018256523, 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1870 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722530+0000", 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722530+0000", 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722530+0000", 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4400000000000001e-07 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722533+0000", 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1629999999999999e-06 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722539+0000", 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1990000000000003e-06 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722545+0000", 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7089999999999999e-06 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740611+0000", 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.018066647000000002 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740687+0000", 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6198999999999993e-05 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740787+0000", 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9161999999999997e-05 2024-09-11T09:30:05.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:1871 3.18 3:1ef2ea64:::rbd_data.11064477c615.000000000000002d:21 [read 1916928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.722551+0000", 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.36788906199999999, 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.018428502999999999, 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:05.179 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1871 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722551+0000", 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722551+0000", 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722552+0000", 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8000000000000001e-07 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722554+0000", 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0999999999999998e-06 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722560+0000", 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1020000000000002e-06 2024-09-11T09:30:05.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.722564+0000", 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6739999999999996e-06 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740817+0000", 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.018252072000000001 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:05.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740902+0000", 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5690000000000001e-05 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.740980+0000", 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7484999999999994e-05 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:05.183 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:05.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:05.186 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:05.184+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:05.186 INFO:teuthology.orchestra.run.smithi107.stdout:4624 write 0xe90e400 thru 0xe9150a6 (0x6ca7 bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4626 read 0x892c00 thru 0x8a1649 (0xea4a bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4627 punch from 0xb2b3200 to 0xb2bd200, (0xa000 bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4628 write 0x7306400 thru 0x730d142 (0x6d43 bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4629 trunc from 0xe915200 to 0x1e07400 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4633 read 0x10b7e00 thru 0x10bdf9b (0x619c bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4634 trunc from 0x1e07400 to 0x41f0e00 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4636 read 0x29f0600 thru 0x29ff051 (0xea52 bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4638 trunc from 0x41f0e00 to 0xd58b800 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4639 write 0x2aa8a00 thru 0x2ab80e9 (0xf6ea bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4642 write 0x9fec800 thru 0x9ff6668 (0x9e69 bytes) 2024-09-11T09:30:05.187 INFO:teuthology.orchestra.run.smithi107.stdout:4643 clone 32 order 23 su 65536 sc 5 2024-09-11T09:30:05.286 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:05.286+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:05.290 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:05.290 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:05.290 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:05.298 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:05.387 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:05.387+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:05.442 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:05.442 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:05.442 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:05.442 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:05.442 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:05.451 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:05.488 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:05.488+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:05.589 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:05.589+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:05.594 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.156768131999996, 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.540862404999999, 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.540664704000001, 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.540406277999999, 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.540376393999999, 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.539843242, 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.538965792999999, 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.538926827000001, 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.538883350999999, 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.538758733999998, 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.538193264, 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.538162627999998, 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.536026196000002, 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.534505708000001, 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.530431878999998, 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.530318085000001, 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.530233403, 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.530187412, 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.529816113999999, 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 18.529674164999999, 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:05.696 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:05.698 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:05.698 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:05.698+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:05.799 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:05.799+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:05.800 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:05.800 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:05.800 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:05.809 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:05.900 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:05.899+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:05.903 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone31 from 0xd58b800 (overlap 0x34a400) to 0x1f9400 2024-09-11T09:30:05.958 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:05.958 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:05.958 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:05.958 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:05.958 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:05.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:06.001 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:06.001+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1846 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1841152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707438773, 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042988987999999999, 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1846 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367077+0000", 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367078+0000", 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367080+0000", 2024-09-11T09:30:06.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2280000000000001e-06 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367096+0000", 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6489000000000001e-05 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367104+0000", 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9489999999999996e-06 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.409993+0000", 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042889161000000002 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410031+0000", 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7270000000000001e-05 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410066+0000", 2024-09-11T09:30:06.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5561000000000001e-05 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1847 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1867776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707401388000001, 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043046174999999999, 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1847 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:06.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367115+0000", 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2300000000000001e-07 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367118+0000", 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9110000000000002e-06 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367125+0000", 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9959999999999996e-06 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367128+0000", 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.281e-06 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410089+0000", 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042960274999999999 2024-09-11T09:30:06.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410124+0000", 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5809000000000002e-05 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410161+0000", 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6380000000000001e-05 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1848 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1894400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707382070000001, 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043121843999999999, 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1848 2024-09-11T09:30:06.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367134+0000", 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8599999999999999e-07 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367136+0000", 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6640000000000001e-06 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367139+0000", 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4819999999999999e-06 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367142+0000", 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5730000000000002e-06 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410184+0000", 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043042149000000002 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410220+0000", 2024-09-11T09:30:06.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5952999999999999e-05 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410256+0000", 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5737e-05 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1849 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1908736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:06.089 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707370824000002, 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043213150999999998, 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1849 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367145+0000", 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3699999999999999e-07 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367147+0000", 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.767e-06 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367152+0000", 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3739999999999997e-06 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367167+0000", 2024-09-11T09:30:06.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4778e-05 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410276+0000", 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043108922000000001 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410312+0000", 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5917000000000002e-05 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410358+0000", 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6156000000000003e-05 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1850 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1935360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707328918999998, 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043269544, 2024-09-11T09:30:06.094 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1850 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367187+0000", 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2e-07 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367190+0000", 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.379e-06 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367195+0000", 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8729999999999998e-06 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367198+0000", 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4709999999999999e-06 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410382+0000", 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043183925999999997 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410420+0000", 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7892000000000002e-05 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410457+0000", 2024-09-11T09:30:06.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6582999999999999e-05 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1851 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1961984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707311515000001, 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043346671000000003, 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.099 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1851 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367204+0000", 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367205+0000", 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367206+0000", 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4079999999999999e-06 2024-09-11T09:30:06.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367211+0000", 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2700000000000004e-06 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367216+0000", 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4399999999999998e-06 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410478+0000", 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262523999999997 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410516+0000", 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7314999999999998e-05 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410551+0000", 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5386000000000003e-05 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1852 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 1976320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707293620000002, 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043422107000000001, 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1852 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367222+0000", 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367223+0000", 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.34e-07 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367225+0000", 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3099999999999999e-06 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367231+0000", 2024-09-11T09:30:06.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9050000000000002e-06 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367236+0000", 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7230000000000004e-06 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410573+0000", 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043337537000000002 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410609+0000", 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5706e-05 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410644+0000", 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5491999999999998e-05 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1853 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2002944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707273440000002, 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043508809000000002, 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1853 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367243+0000", 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1399999999999997e-07 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367245+0000", 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1610000000000001e-06 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367252+0000", 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367256+0000", 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410668+0000", 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043411980000000003 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410714+0000", 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5878999999999999e-05 2024-09-11T09:30:06.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410751+0000", 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6956999999999999e-05 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1854 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2029568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:30:06.113 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707252455999999, 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043580835999999998, 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1854 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367263+0000", 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367264+0000", 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367271+0000", 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6020000000000002e-06 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367278+0000", 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3389999999999998e-06 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367284+0000", 2024-09-11T09:30:06.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7030000000000003e-06 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410773+0000", 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043489130000000001 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410809+0000", 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6355e-05 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410844+0000", 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5321999999999998e-05 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1855 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2043904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707225358999999, 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043657658000000002, 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.118 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1855 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:06.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367291+0000", 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.58e-07 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367293+0000", 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.922e-06 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367299+0000", 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8390000000000003e-06 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367304+0000", 2024-09-11T09:30:06.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410873+0000", 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043569496999999999 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410911+0000", 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8371000000000002e-05 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410948+0000", 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6801000000000002e-05 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1856 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2070528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707206071999998, 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043730514999999998, 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1856 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367310+0000", 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367484+0000", 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017401600000000001 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367488+0000", 2024-09-11T09:30:06.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6260000000000001e-06 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367497+0000", 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4250000000000004e-06 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367504+0000", 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8399999999999997e-06 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.410970+0000", 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043466295000000002 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411007+0000", 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.65e-05 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411040+0000", 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3813000000000003e-05 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.127 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1857 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2111488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.707001712, 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043619054999999997, 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1857 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.367514+0000", 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368016+0000", 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000501542 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368020+0000", 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2370000000000003e-06 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368030+0000", 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0030999999999999e-05 2024-09-11T09:30:06.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368037+0000", 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7859999999999999e-06 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411063+0000", 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043026139999999997 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411100+0000", 2024-09-11T09:30:06.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7091000000000001e-05 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411133+0000", 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3228000000000001e-05 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1858 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2138112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:06.132 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.706467955999997, 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043175696, 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1858 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368048+0000", 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6800000000000001e-07 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368051+0000", 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2970000000000002e-06 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368067+0000", 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6458999999999999e-05 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368087+0000", 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0183e-05 2024-09-11T09:30:06.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411155+0000", 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043067691999999998 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411191+0000", 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5834000000000003e-05 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411224+0000", 2024-09-11T09:30:06.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2762999999999998e-05 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1859 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2179072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.706419062999998, 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043216077999999998, 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1859 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368097+0000", 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4700000000000001e-07 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368100+0000", 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.667e-06 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368108+0000", 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9549999999999992e-06 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368113+0000", 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2020000000000003e-06 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411245+0000", 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043131544000000001 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411280+0000", 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5670999999999998e-05 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411313+0000", 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2492e-05 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1860 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2205696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.706370974999999, 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043262854000000003, 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1860 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.142 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368145+0000", 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368146+0000", 2024-09-11T09:30:06.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4899999999999995e-07 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368149+0000", 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7299999999999999e-06 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368158+0000", 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6680000000000007e-06 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368172+0000", 2024-09-11T09:30:06.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3799e-05 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411335+0000", 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043162813000000001 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411371+0000", 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6152999999999998e-05 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411408+0000", 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7141999999999998e-05 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1861 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2246656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.706332936999999, 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043320775999999998, 2024-09-11T09:30:06.146 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1861 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368183+0000", 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368379+0000", 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019558800000000001 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368383+0000", 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2409999999999997e-06 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368391+0000", 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1270000000000003e-06 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368397+0000", 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.268e-06 2024-09-11T09:30:06.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411427+0000", 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043029369999999997 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411466+0000", 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9879000000000002e-05 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411504+0000", 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7302999999999998e-05 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1862 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2273280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.706109095000002, 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043194679999999999, 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1862 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368407+0000", 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368554+0000", 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014742999999999999 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368558+0000", 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3730000000000001e-06 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368569+0000", 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1517e-05 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368575+0000", 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.801e-06 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411526+0000", 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042950849999999999 2024-09-11T09:30:06.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411566+0000", 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9749000000000002e-05 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411602+0000", 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5960000000000001e-05 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1863 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2314240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.705931993, 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043117299999999997, 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1863 2024-09-11T09:30:06.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368584+0000", 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368704+0000", 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012039799999999999 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368708+0000", 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1980000000000001e-06 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368715+0000", 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7060000000000004e-06 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368721+0000", 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6529999999999999e-06 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411623+0000", 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042902402999999999 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411663+0000", 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9773e-05 2024-09-11T09:30:06.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411701+0000", 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8169e-05 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1864 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2340864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.705786437999997, 2024-09-11T09:30:06.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043070293000000003, 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1864 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:30:06.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368730+0000", 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368826+0000", 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6415000000000004e-05 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368829+0000", 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.191e-06 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368837+0000", 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5220000000000001e-06 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368842+0000", 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6559999999999997e-06 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411723+0000", 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042880670000000003 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411763+0000", 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0107000000000001e-05 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411800+0000", 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6732e-05 2024-09-11T09:30:06.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4728.0:1865 3.3 3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9 [read 2381824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 62.705665052000001, 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.043056839, 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4728", 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/983648617", 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1865 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.368851+0000", 2024-09-11T09:30:06.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369594+0000", 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00074339700000000002 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369599+0000", 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9520000000000002e-06 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369610+0000", 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0645999999999999e-05 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.369616+0000", 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411823+0000", 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.042206434000000001 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411868+0000", 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5639999999999997e-05 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:03.411908+0000", 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9298000000000002e-05 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.169 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:06.171 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:06.171 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:06.171+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:06.271 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:06.272+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:06.315 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:06.316 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:06.316 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:06.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:06.372 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:06.373+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:06.431 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:06.431 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:06.432 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:06.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:06.482 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-11T09:30:06.482 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:06.482+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:06.574 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:06.574+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.619977364999997, 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:06.604 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:06.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.619931321000003, 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:06.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.619919651000004, 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:06.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:06.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618631381999997, 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:06.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:06.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.630 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618595681000002, 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:06.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:06.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618573644000001, 2024-09-11T09:30:06.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:06.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:06.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618538059000002, 2024-09-11T09:30:06.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:06.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:06.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618477716000001, 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:06.645 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:06.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618429032000002, 2024-09-11T09:30:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:06.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:06.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:06.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:06.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:06.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:06.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:06.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:06.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:06.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:06.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:06.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:06.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:06.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:06.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:06.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618372567999998, 2024-09-11T09:30:06.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.665 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:06.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:06.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:06.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:06.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:06.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:06.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:06.675 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618353012999997, 2024-09-11T09:30:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.676 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:06.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:06.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618305249999999, 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:06.681 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:06.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:06.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618191512000003, 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:06.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:06.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618172768000001, 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:06.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618133778999997, 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:06.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:06.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:06.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.618110672, 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:06.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.910034181, 2024-09-11T09:30:06.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:06.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.547634851000002, 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:06.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:06.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.542383774000001, 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:06.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.542123404000002, 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.724 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:06.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:06.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:06.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:06.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:06.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:06.730 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:06.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:06.732 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:06.732+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:06.732 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:30:06.735 INFO:teuthology.orchestra.run.smithi107.stdout:4645 write 0x6a46400 thru 0x6a52fd0 (0xcbd1 bytes) 2024-09-11T09:30:06.736 INFO:teuthology.orchestra.run.smithi107.stdout:4646 punch from 0xa180600 to 0xa18be00, (0xb800 bytes) 2024-09-11T09:30:06.763 INFO:teuthology.orchestra.run.smithi107.stdout:4647 punch from 0xa6e3200 to 0xa6f2000, (0xee00 bytes) 2024-09-11T09:30:06.775 INFO:teuthology.orchestra.run.smithi107.stdout:4648 write 0x278dc00 thru 0x2796182 (0x8583 bytes) 2024-09-11T09:30:06.776 INFO:teuthology.orchestra.run.smithi107.stdout:4650 read 0xb026000 thru 0xb031bd8 (0xbbd9 bytes) 2024-09-11T09:30:06.790 INFO:teuthology.orchestra.run.smithi107.stdout:4651 trunc from 0xd58b800 to 0x107e600 2024-09-11T09:30:06.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:30:06.833 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:06.832+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:4654 write 0x4d02400 thru 0x4d0c0de (0x9cdf bytes) 2024-09-11T09:30:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:4655 punch from 0x19fb000 to 0x19fec00, (0x3c00 bytes) 2024-09-11T09:30:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:4656 read 0x3595a00 thru 0x35a0361 (0xa962 bytes) 2024-09-11T09:30:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:4659 read 0x38cba00 thru 0x38dadf1 (0xf3f2 bytes) 2024-09-11T09:30:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:4661 read 0x2f8ce00 thru 0x2f958be (0x8abf bytes) 2024-09-11T09:30:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:4663 write 0x2229600 thru 0x223029d (0x6c9e bytes) 2024-09-11T09:30:06.868 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:06.869 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:06.869 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:06.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:30:06.884 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:30:06.885 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:30:06.888 INFO:teuthology.orchestra.run.smithi107.stdout:4669 write 0xc46a400 thru 0xc472a68 (0x8669 bytes) 2024-09-11T09:30:06.888 INFO:teuthology.orchestra.run.smithi107.stdout:4670 write 0x6b6a400 thru 0x6b7843c (0xe03d bytes) 2024-09-11T09:30:06.888 INFO:teuthology.orchestra.run.smithi107.stdout:4672 punch from 0x452fe00 to 0x4536800, (0x6a00 bytes) 2024-09-11T09:30:06.888 INFO:teuthology.orchestra.run.smithi107.stdout:4674 read 0xa4d3e00 thru 0xa4d7d38 (0x3f39 bytes) 2024-09-11T09:30:06.889 INFO:teuthology.orchestra.run.smithi107.stdout:4675 write 0x9bf2800 thru 0x9bf84ed (0x5cee bytes) 2024-09-11T09:30:06.889 INFO:teuthology.orchestra.run.smithi107.stdout:4676 write 0xb4c8600 thru 0xb4cdbad (0x55ae bytes) 2024-09-11T09:30:06.889 INFO:teuthology.orchestra.run.smithi107.stdout:4677 write 0x3b66e00 thru 0x3b76b3b (0xfd3c bytes) 2024-09-11T09:30:06.889 INFO:teuthology.orchestra.run.smithi107.stdout:4678 read 0xa169e00 thru 0xa177629 (0xd82a bytes) 2024-09-11T09:30:06.889 INFO:teuthology.orchestra.run.smithi107.stdout:4679 write 0x9feaa00 thru 0x9ff5c89 (0xb28a bytes) 2024-09-11T09:30:06.890 INFO:teuthology.orchestra.run.smithi107.stdout:4681 read 0x819e800 thru 0x81a074b (0x1f4c bytes) 2024-09-11T09:30:06.890 INFO:teuthology.orchestra.run.smithi107.stdout:4685 read 0x9735200 thru 0x973a22d (0x502e bytes) 2024-09-11T09:30:06.890 INFO:teuthology.orchestra.run.smithi107.stdout:4686 read 0x94dc800 thru 0x94eb243 (0xea44 bytes) 2024-09-11T09:30:06.892 INFO:teuthology.orchestra.run.smithi107.stdout:4687 trunc from 0xc472c00 to 0x82dba00 2024-09-11T09:30:06.909 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:06.917 INFO:teuthology.orchestra.run.smithi107.stdout:4688 read 0xb11800 thru 0xb20586 (0xed87 bytes) 2024-09-11T09:30:06.919 INFO:teuthology.orchestra.run.smithi107.stdout:4689 punch from 0x62a3600 to 0x62a9c00, (0x6600 bytes) 2024-09-11T09:30:06.919 INFO:teuthology.orchestra.run.smithi107.stdout:4691 punch from 0x57b4000 to 0x57b9400, (0x5400 bytes) 2024-09-11T09:30:06.919 INFO:teuthology.orchestra.run.smithi107.stdout:4692 read 0x6bd1a00 thru 0x6bd59b9 (0x3fba bytes) 2024-09-11T09:30:06.919 INFO:teuthology.orchestra.run.smithi107.stdout:4694 write 0x76b0000 thru 0x76bbe08 (0xbe09 bytes) 2024-09-11T09:30:06.919 INFO:teuthology.orchestra.run.smithi107.stdout:4695 read 0x46abc00 thru 0x46ba103 (0xe504 bytes) 2024-09-11T09:30:06.930 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:06.930 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:06.934 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:06.933+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:06.940 INFO:teuthology.orchestra.run.smithi107.stdout:4698 write 0xb4fc400 thru 0xb4fe460 (0x2061 bytes) 2024-09-11T09:30:06.941 INFO:teuthology.orchestra.run.smithi107.stdout:4701 clone 33 order 19 su 2048 sc 7 2024-09-11T09:30:06.950 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:06.950 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:06.968 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:06.968 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:06.985 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:06.986 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:07.012 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:30:07.013 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:07.013 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-11T09:30:07.013 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T09:30:07.013 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T09:30:07.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-11T09:30:07.026 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.026 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.034 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:07.034+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:07.035 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:30:07.115 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:07.115 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T09:30:07.115 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:07.135 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:07.135+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:07.182 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.183 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.191 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:07.275 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:07.276+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:07.337 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:07.338+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:07.338 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.338 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.347 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:07.438 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:07.438+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:07.494 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.494 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.503 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:07.546 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:07.546+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:07.610 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.610 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:07.659 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:07.660+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:07.741 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:07.740+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:07.766 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.766 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:07.831 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:30:07.841 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:07.841+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:07.930 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:07.930 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:07.938 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone32 from 0xb4fe600 (overlap 0x107e600) to 0x698e00 2024-09-11T09:30:07.939 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:07.941 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:07.942+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:08.044 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:08.044+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:08.050 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.050 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.060 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:08.146 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:08.145+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:08.165 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.165 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:08.248 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:08.249+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:08.292 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.292 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:08.303 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-11T09:30:08.373 INFO:teuthology.orchestra.run.smithi107.stdout:4703 write 0x9fac800 thru 0x9fb08ee (0x40ef bytes) 2024-09-11T09:30:08.377 INFO:teuthology.orchestra.run.smithi107.stdout:4704 trunc from 0xb4fe600 to 0x7fe0e00 2024-09-11T09:30:08.385 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:08.386+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:08.450 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:08.450+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:08.483 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.483 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.495 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:08.550 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:08.550+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:08.598 INFO:teuthology.orchestra.run.smithi107.stdout:4707 write 0x3f71e00 thru 0x3f7ebbd (0xcdbe bytes) 2024-09-11T09:30:08.652 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:08.651+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:08.672 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.672 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.684 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:08.701 INFO:teuthology.orchestra.run.smithi107.stdout:4710 write 0xd88a400 thru 0xd88d227 (0x2e28 bytes) 2024-09-11T09:30:08.707 INFO:teuthology.orchestra.run.smithi107.stdout:4712 read 0x233de00 thru 0x233edd2 (0xfd3 bytes) 2024-09-11T09:30:08.718 INFO:teuthology.orchestra.run.smithi107.stdout:4713 trunc from 0xd88d400 to 0x5e89000 2024-09-11T09:30:08.753 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:08.753+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:08.814 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.814 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:08.830 INFO:teuthology.orchestra.run.smithi107.stdout:4714 trunc from 0x5e89000 to 0x51efc00 2024-09-11T09:30:08.854 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:08.855+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:08.955 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:08.955+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:08.958 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:08.958 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:08.959 INFO:teuthology.orchestra.run.smithi107.stdout:4718 trunc from 0x51efc00 to 0xc2bd400 2024-09-11T09:30:08.969 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:08.970 INFO:teuthology.orchestra.run.smithi107.stdout:4720 read 0x9bace00 thru 0x9bb1eb9 (0x50ba bytes) 2024-09-11T09:30:08.976 INFO:teuthology.orchestra.run.smithi107.stdout:4721 write 0x88f3200 thru 0x88fbdfd (0x8bfe bytes) 2024-09-11T09:30:08.976 INFO:teuthology.orchestra.run.smithi107.stdout:4723 read 0x3c77200 thru 0x3c77e02 (0xc03 bytes) 2024-09-11T09:30:08.981 INFO:teuthology.orchestra.run.smithi107.stdout:4724 trunc from 0xc2bd400 to 0x81be200 2024-09-11T09:30:08.997 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:09.031 INFO:teuthology.orchestra.run.smithi107.stdout:4725 write 0xc893c00 thru 0xc898853 (0x4c54 bytes) 2024-09-11T09:30:09.040 INFO:teuthology.orchestra.run.smithi107.stdout:4727 trunc from 0xc898a00 to 0xeaea400 2024-09-11T09:30:09.055 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:09.056+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:09.074 INFO:teuthology.orchestra.run.smithi107.stdout:4728 trunc from 0xeaea400 to 0x9105000 2024-09-11T09:30:09.091 INFO:teuthology.orchestra.run.smithi107.stdout:4729 read 0x764da00 thru 0x765ca9c (0xf09d bytes) 2024-09-11T09:30:09.095 INFO:teuthology.orchestra.run.smithi107.stdout:4730 trunc from 0x9105000 to 0xb9f4600 2024-09-11T09:30:09.118 INFO:teuthology.orchestra.run.smithi107.stdout:4731 write 0x9d3dc00 thru 0x9d44fcc (0x73cd bytes) 2024-09-11T09:30:09.124 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:30:09.124 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:30:09.125 INFO:teuthology.orchestra.run.smithi107.stdout:4733 write 0x3f85000 thru 0x3f8520c (0x20d bytes) 2024-09-11T09:30:09.125 INFO:teuthology.orchestra.run.smithi107.stdout:4734 read 0x5eca600 thru 0x5ed0c49 (0x664a bytes) 2024-09-11T09:30:09.125 INFO:teuthology.orchestra.run.smithi107.stdout:4735 read 0x42ae200 thru 0x42b8d94 (0xab95 bytes) 2024-09-11T09:30:09.129 INFO:teuthology.orchestra.run.smithi107.stdout:4736 read 0x8929800 thru 0x892db93 (0x4394 bytes) 2024-09-11T09:30:09.129 INFO:teuthology.orchestra.run.smithi107.stdout:4747 write 0x741fa00 thru 0x742b17d (0xb77e bytes) 2024-09-11T09:30:09.129 INFO:teuthology.orchestra.run.smithi107.stdout:4750 read 0x7c85600 thru 0x7c87f08 (0x2909 bytes) 2024-09-11T09:30:09.137 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:09.137 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:09.141 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:09.141 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:09.142 INFO:teuthology.orchestra.run.smithi107.stdout:4753 trunc from 0xb9f4600 to 0xca5400 2024-09-11T09:30:09.147 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:09.156 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:09.157+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:09.162 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:09.162 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:09.184 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:09.184 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:09.212 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:09.212 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:09.248 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:09.248 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:09.257 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:09.258+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:09.307 INFO:teuthology.orchestra.run.smithi107.stdout:4755 punch from 0x8f8200 to 0x8fe600, (0x6400 bytes) 2024-09-11T09:30:09.339 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:09.339 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:09.339 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:09.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:09.393 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:09.394+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:09.412 INFO:teuthology.orchestra.run.smithi107.stdout:4757 write 0x8e84a00 thru 0x8e869a9 (0x1faa bytes) 2024-09-11T09:30:09.414 INFO:teuthology.orchestra.run.smithi107.stdout:4758 read 0x703ec00 thru 0x70431a2 (0x45a3 bytes) 2024-09-11T09:30:09.414 INFO:teuthology.orchestra.run.smithi107.stdout:4761 punch from 0x537ba00 to 0x538a000, (0xe600 bytes) 2024-09-11T09:30:09.422 INFO:teuthology.orchestra.run.smithi107.stdout:4762 trunc from 0x8e86a00 to 0xe919400 2024-09-11T09:30:09.445 INFO:teuthology.orchestra.run.smithi107.stdout:4764 write 0xb3c0000 thru 0xb3cbb2b (0xbb2c bytes) 2024-09-11T09:30:09.446 INFO:teuthology.orchestra.run.smithi107.stdout:4765 write 0x3831400 thru 0x383dc8c (0xc88d bytes) 2024-09-11T09:30:09.446 INFO:teuthology.orchestra.run.smithi107.stdout:4766 punch from 0xb64a400 to 0xb653600, (0x9200 bytes) 2024-09-11T09:30:09.447 INFO:teuthology.orchestra.run.smithi107.stdout:4767 clone 34 order 22 su 65536 sc 8 2024-09-11T09:30:09.459 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:09.460+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:09.519 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:09.520 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:09.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:09.520 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:09.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.520 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.521 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:09.521 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.535916554000003, 2024-09-11T09:30:09.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:09.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.535870510000002, 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.527 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:09.531 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.535858840000003, 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:09.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:09.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.536 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534570571000003, 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:09.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:09.541 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534534870000002, 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.542 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:09.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.546 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534512833000001, 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:09.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:09.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:09.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534477248000002, 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:09.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:09.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534416905, 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:09.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534368221000001, 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:09.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:09.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534311756999998, 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:09.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:09.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:09.569 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534292202000003, 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:09.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534244438999998, 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:09.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:09.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:09.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534130701000002, 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.578 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:09.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:09.582 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534111957, 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534072967999997, 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:09.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 58.534049861, 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:09.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:09.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 39.82597337, 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.597 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:09.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.463574040000001, 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:09.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:09.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.458322963000001, 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:09.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 22.458062593000001, 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:09.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:09.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:09.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:09.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:09.619 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:09.620 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:09.621 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:30:09.622 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:09.621+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:09.722 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:09.722+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:09.729 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:09.730 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:09.730 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:09.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-11T09:30:09.823 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:09.823+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:09.847 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:30:09.889 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:09.889 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:09.889 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:09.899 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-11T09:30:09.923 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:09.924+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:09.936 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone33 from 0xe919400 (overlap 0xca5400) to 0xb61800 2024-09-11T09:30:10.024 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:10.024+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:10.057 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:10.057 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1768 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 194560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.180484561, 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024959592999999999, 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1768 2024-09-11T09:30:10.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876728+0000", 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876729+0000", 2024-09-11T09:30:10.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.82e-07 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876731+0000", 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.933e-06 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876739+0000", 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9650000000000008e-06 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876744+0000", 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3229999999999997e-06 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901571+0000", 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024827384000000001 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901622+0000", 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0945000000000001e-05 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901688+0000", 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5561000000000005e-05 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1769 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.180462276, 2024-09-11T09:30:10.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025169627999999999, 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1769 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:30:10.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876750+0000", 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876751+0000", 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.34e-07 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876756+0000", 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0830000000000003e-06 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876762+0000", 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4130000000000003e-06 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876767+0000", 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0749999999999997e-06 2024-09-11T09:30:10.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901719+0000", 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024951026000000001 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901809+0000", 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0277000000000003e-05 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901920+0000", 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011132 2024-09-11T09:30:10.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1770 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 239616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.180438835999993, 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025364301999999998, 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:10.067 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1770 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.068 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876774+0000", 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2800000000000002e-07 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876776+0000", 2024-09-11T09:30:10.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9400000000000001e-06 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876783+0000", 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4409999999999998e-06 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876788+0000", 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9640000000000002e-06 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.901950+0000", 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025162773999999999 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902029+0000", 2024-09-11T09:30:10.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8350999999999996e-05 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902138+0000", 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109404 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1771 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 262144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:10.072 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.180418989000003, 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025587322999999999, 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1771 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876794+0000", 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.53e-07 2024-09-11T09:30:10.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876806+0000", 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1454000000000001e-05 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876813+0000", 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.9130000000000003e-06 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876819+0000", 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.327e-06 2024-09-11T09:30:10.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902176+0000", 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025357111000000002 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902269+0000", 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3506999999999998e-05 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902381+0000", 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00011165799999999999 2024-09-11T09:30:10.076 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4663.0:1772 3.12 3:4db47056:::rbd_data.1106cda8c42d.0000000000000006:a [read 284672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.180386612000007, 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025772489999999999, 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4663", 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4201579336", 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1772 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876826+0000", 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876827+0000", 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6400000000000002e-07 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876829+0000", 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.238e-06 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876835+0000", 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5089999999999999e-06 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.876844+0000", 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.833e-06 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902409+0000", 2024-09-11T09:30:10.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025565195999999998 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902503+0000", 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3205e-05 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:05.902599+0000", 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5945000000000004e-05 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:548 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.030892640000001, 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026786889000000001, 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.082 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 548 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026320+0000", 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026321+0000", 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8599999999999996e-07 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026326+0000", 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2669999999999997e-06 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026342+0000", 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5695000000000001e-05 2024-09-11T09:30:10.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026350+0000", 2024-09-11T09:30:10.085 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.371e-06 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026418+0000", 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7268999999999996e-05 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026621+0000", 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000203172 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053107+0000", 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026486129000000001 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:558 3.1f 3:fa19977a:::rbd_data.1106b643ff7f.000000000000001a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.030750074, 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026276384, 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 558 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026463+0000", 2024-09-11T09:30:10.091 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1200000000000002e-07 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026468+0000", 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2729999999999996e-06 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026508+0000", 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0095000000000002e-05 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.092 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026516+0000", 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6549999999999993e-06 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026689+0000", 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017269599999999999 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026791+0000", 2024-09-11T09:30:10.093 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010203 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.052739+0000", 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025947923000000001 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:565 3.1f 3:fa508b4a:::rbd_data.1106b643ff7f.000000000000001e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.030672792000001, 2024-09-11T09:30:10.094 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026702457999999998, 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 565 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.095 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026540+0000", 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.096 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026541+0000", 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9999999999999997e-07 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026546+0000", 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6829999999999996e-06 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026554+0000", 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6869999999999994e-06 2024-09-11T09:30:10.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026559+0000", 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.098 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027294+0000", 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000734361 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027468+0000", 2024-09-11T09:30:10.099 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017458400000000001 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053242+0000", 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025774047000000001 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.100 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:595 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:30:10.101 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.030262645000001, 2024-09-11T09:30:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026285078, 2024-09-11T09:30:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.102 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 595 2024-09-11T09:30:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.103 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:30:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.104 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026950+0000", 2024-09-11T09:30:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.105 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026951+0000", 2024-09-11T09:30:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3799999999999997e-07 2024-09-11T09:30:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.106 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026954+0000", 2024-09-11T09:30:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0599999999999999e-06 2024-09-11T09:30:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026961+0000", 2024-09-11T09:30:10.107 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.3189999999999997e-06 2024-09-11T09:30:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.026967+0000", 2024-09-11T09:30:10.108 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5890000000000002e-06 2024-09-11T09:30:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027011+0000", 2024-09-11T09:30:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4682999999999999e-05 2024-09-11T09:30:10.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027107+0000", 2024-09-11T09:30:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5419000000000002e-05 2024-09-11T09:30:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053235+0000", 2024-09-11T09:30:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026128470000000001 2024-09-11T09:30:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:629 3.1f 3:fb0b56ac:::rbd_data.1106b643ff7f.0000000000000048:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.029829385999999, 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.02599951, 2024-09-11T09:30:10.112 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.113 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 629 2024-09-11T09:30:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:30:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027383+0000", 2024-09-11T09:30:10.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027384+0000", 2024-09-11T09:30:10.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.23e-07 2024-09-11T09:30:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027391+0000", 2024-09-11T09:30:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0899999999999999e-06 2024-09-11T09:30:10.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027408+0000", 2024-09-11T09:30:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.717e-05 2024-09-11T09:30:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027416+0000", 2024-09-11T09:30:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1570000000000003e-06 2024-09-11T09:30:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027791+0000", 2024-09-11T09:30:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00037519499999999999 2024-09-11T09:30:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027902+0000", 2024-09-11T09:30:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000111215 2024-09-11T09:30:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053383+0000", 2024-09-11T09:30:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025480860000000001 2024-09-11T09:30:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:638 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.029724792, 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025900833000000002, 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.123 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 638 2024-09-11T09:30:10.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027488+0000", 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027489+0000", 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0299999999999998e-07 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027492+0000", 2024-09-11T09:30:10.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6270000000000002e-06 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027501+0000", 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8440000000000004e-06 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.027507+0000", 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0689999999999998e-06 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028168+0000", 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00066121099999999998 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028276+0000", 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107435 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053389+0000", 2024-09-11T09:30:10.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025112944000000002 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:660 3.12 3:4f6d0c71:::rbd_data.1106b643ff7f.000000000000005c:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.029067605000002, 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025370475, 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.129 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 660 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028145+0000", 2024-09-11T09:30:10.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028146+0000", 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2900000000000004e-07 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028149+0000", 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.9000000000000002e-06 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028156+0000", 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2200000000000003e-06 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028161+0000", 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6910000000000002e-06 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028580+0000", 2024-09-11T09:30:10.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000418625 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.028685+0000", 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010467300000000001 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.053516+0000", 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024830837000000001 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1000 3.12 3:49cfef73:::rbd_data.1106b643ff7f.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.002230702999999, 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037281880000000003, 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.134 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1000 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054982+0000", 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054983+0000", 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8599999999999998e-07 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054987+0000", 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1670000000000001e-06 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.054996+0000", 2024-09-11T09:30:10.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7490000000000004e-06 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055002+0000", 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4799999999999998e-06 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055397+0000", 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00039521700000000001 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055474+0000", 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6872000000000005e-05 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055665+0000", 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000191013 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056748+0000", 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010830779999999999 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092163+0000", 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035415042000000001 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092225+0000", 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1555000000000003e-05 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092264+0000", 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9020999999999998e-05 2024-09-11T09:30:10.139 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1010 3.12 3:4b35ab0d:::rbd_data.1106b643ff7f.000000000000004e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.002073752000001, 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.037320416000000002, 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.140 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1010 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:10.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055139+0000", 2024-09-11T09:30:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2199999999999999e-07 2024-09-11T09:30:10.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055142+0000", 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5620000000000002e-06 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055148+0000", 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2400000000000004e-06 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.055153+0000", 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.977e-06 2024-09-11T09:30:10.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056168+0000", 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0010149040000000001 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056237+0000", 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8606e-05 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056413+0000", 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017639199999999999 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057731+0000", 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001317616 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092370+0000", 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034638872000000001 2024-09-11T09:30:10.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092419+0000", 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8989999999999997e-05 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092459+0000", 2024-09-11T09:30:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0834999999999998e-05 2024-09-11T09:30:10.149 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.150 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1018 3.12 3:4c5b4e7d:::rbd_data.1106b643ff7f.0000000000000031:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.000225294, 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.035657162999999999, 2024-09-11T09:30:10.151 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1018 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056987+0000", 2024-09-11T09:30:10.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056988+0000", 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.95e-07 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.056994+0000", 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4500000000000003e-06 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057005+0000", 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1522000000000001e-05 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057013+0000", 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6599999999999995e-06 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057058+0000", 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:10.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057153+0000", 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5377000000000005e-05 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:10.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057338+0000", 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018502199999999999 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.057925+0000", 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00058726299999999998 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092552+0000", 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.034627003000000003 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092598+0000", 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5989000000000003e-05 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092645+0000", 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6431999999999999e-05 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5075.0:1086 3.12 3:4b95dab8:::rbd_data.1106b643ff7f.0000000000000085:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 22.996571789000001, 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.032140305000000001, 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.160 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5075", 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 1086 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060641+0000", 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060642+0000", 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.4300000000000002e-07 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060650+0000", 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1969999999999996e-06 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060660+0000", 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0471e-05 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060667+0000", 2024-09-11T09:30:10.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4829999999999996e-06 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060798+0000", 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00013138599999999999 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.060884+0000", 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5348999999999999e-05 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061065+0000", 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018090700000000001 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.061845+0000", 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00078057400000000002 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092700+0000", 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.030854940000000001 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092749+0000", 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9101999999999998e-05 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:47.092781+0000", 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2053e-05 2024-09-11T09:30:10.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5323.0:832 3.18 3:18760478:::rbd_header.1106a471d92f:head [call rbd.snapshot_add in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.291275035999998, 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3939045349999999, 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5323", 2024-09-11T09:30:10.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2665032934", 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 832 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765938+0000", 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765940+0000", 2024-09-11T09:30:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.48e-06 2024-09-11T09:30:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765952+0000", 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1708999999999999e-05 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765970+0000", 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.8459999999999999e-05 2024-09-11T09:30:10.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.765984+0000", 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4056999999999999e-05 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766066+0000", 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.1976000000000003e-05 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766075+0000", 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2940000000000006e-06 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768269+0000", 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021941510000000001 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768283+0000", 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3966e-05 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.157333+0000", 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3890506730000001 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.157618+0000", 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000284553 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 2", 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158170+0000", 2024-09-11T09:30:10.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00055215699999999995 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159321+0000", 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0011509669999999999 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159683+0000", 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00036161199999999998 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159755+0000", 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1730999999999995e-05 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159842+0000", 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7749000000000003e-05 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5323.0:833 3.18 3:18760478:::rbd_header.1106a471d92f:head [watch reconnect cookie 140016404064176 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.291195827999999, 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.395538188, 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5323", 2024-09-11T09:30:10.176 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2665032934", 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 833 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766017+0000", 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766018+0000", 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6199999999999996e-07 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766023+0000", 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8239999999999999e-06 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766034+0000", 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1231e-05 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766042+0000", 2024-09-11T09:30:10.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6720000000000004e-06 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766151+0000", 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000109078 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766156+0000", 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.6729999999999997e-06 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768984+0000", 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0028281130000000002 2024-09-11T09:30:10.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.768994+0000", 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0012e-05 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158796+0000", 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.389801721 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.158907+0000", 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000110986 2024-09-11T09:30:10.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159892+0000", 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00098477999999999999 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159989+0000", 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7326999999999998e-05 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.161555+0000", 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0015659090000000001 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:16052 3.18 3:18760478:::rbd_header.1106a471d92f:head [watch reconnect cookie 140373288956208 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.291158258999999, 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3957736299999999, 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 16052 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766055+0000", 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766056+0000", 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.7460000000000001e-06 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766063+0000", 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7499999999999997e-06 2024-09-11T09:30:10.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766078+0000", 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.4866e-05 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766087+0000", 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4180000000000003e-06 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766271+0000", 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018336900000000001 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op must wait for map", 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.766278+0000", 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9009999999999997e-06 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.769020+0000", 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0027424229999999999 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:51.769028+0000", 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7190000000000001e-06 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159133+0000", 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3901048120000001 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for rw locks", 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.159234+0000", 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101895 2024-09-11T09:30:10.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160503+0000", 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001268868 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.160624+0000", 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000120931 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:53.161828+0000", 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001203932 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5539.0:2276 3.1b 3:d864a4b2:::rbd_data.1106aa24fca5.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 18874368~3145728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:04.745525+0000", 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 5.3116880589999997, 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.029322306999999999, 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.193 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5539", 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1445759952", 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2276 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.745525+0000", 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.745525+0000", 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.745526+0000", 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.646e-06 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.758667+0000", 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.013141026 2024-09-11T09:30:10.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.758689+0000", 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1331e-05 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.758698+0000", 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0250000000000008e-06 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.758755+0000", 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7397e-05 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.758933+0000", 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00017799800000000001 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.759754+0000", 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00082059099999999996 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.766939+0000", 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0071851290000000002 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.774769+0000", 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0078300139999999997 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.774807+0000", 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8315999999999997e-05 2024-09-11T09:30:10.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:04.774847+0000", 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9833999999999998e-05 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.199 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:10.201 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:10.201 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:10.201+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:10.301 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:10.302+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:10.349 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:10.349 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:10.349 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:10.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:10.402 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:10.402+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:10.502 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:10.503+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:10.510 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:10.510 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:10.510 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:10.518 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:10.547 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-11T09:30:10.603 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:10.604+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:10.629 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.718458568999999, 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.630 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:10.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.718227997, 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.635 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:10.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.639 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.718130793, 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:10.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:10.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.717896029000002, 2024-09-11T09:30:10.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:10.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:10.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.717700012999998, 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.649 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:10.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.717431576000003, 2024-09-11T09:30:10.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:10.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.717329825999997, 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:10.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.716936169, 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.662 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:10.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.666 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.716809924000003, 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:10.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.671 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.716600542000002, 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.672 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:10.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:10.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.676 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.716489592999999, 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.677 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:10.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.716233655000003, 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:10.682 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:10.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:10.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.716081125999999, 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:10.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:10.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.715888761999999, 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.692 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:10.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.715558344000002, 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:10.696 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:10.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:10.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:10.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.701 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.715282014000003, 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.702 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:10.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:10.706 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.715228093999997, 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.707 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:10.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:10.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.715027524999996, 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:10.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:10.716 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.714671985999999, 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:10.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:10.721 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.935406452999999, 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:10.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:10.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:10.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:10.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:10.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:10.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:10.728 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:10.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:10.731 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:10.729+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:10.731 INFO:teuthology.orchestra.run.smithi107.stdout:4770 write 0x5509200 thru 0x5517c7f (0xea80 bytes) 2024-09-11T09:30:10.731 INFO:teuthology.orchestra.run.smithi107.stdout:4771 punch from 0x8786a00 to 0x8790a00, (0xa000 bytes) 2024-09-11T09:30:10.731 INFO:teuthology.orchestra.run.smithi107.stdout:4774 punch from 0x3305000 to 0x330a400, (0x5400 bytes) 2024-09-11T09:30:10.731 INFO:teuthology.orchestra.run.smithi107.stdout:4778 read 0x6150600 thru 0x615c1ce (0xbbcf bytes) 2024-09-11T09:30:10.732 INFO:teuthology.orchestra.run.smithi107.stdout:4779 read 0x88da00 thru 0x89cde8 (0xf3e9 bytes) 2024-09-11T09:30:10.732 INFO:teuthology.orchestra.run.smithi107.stdout:4780 trunc from 0xe919400 to 0x335b800 2024-09-11T09:30:10.732 INFO:teuthology.orchestra.run.smithi107.stdout:4781 read 0xed8a00 thru 0xee3332 (0xa933 bytes) 2024-09-11T09:30:10.732 INFO:teuthology.orchestra.run.smithi107.stdout:4782 read 0x28d6400 thru 0x28e1bb5 (0xb7b6 bytes) 2024-09-11T09:30:10.732 INFO:teuthology.orchestra.run.smithi107.stdout:4784 punch from 0xb17c00 to 0xb27a00, (0xfe00 bytes) 2024-09-11T09:30:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:4785 trunc from 0x335b800 to 0xd814600 2024-09-11T09:30:10.797 INFO:teuthology.orchestra.run.smithi107.stdout:4787 trunc from 0xd814600 to 0x19f8600 2024-09-11T09:30:10.830 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:10.830+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:10.851 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:30:10.917 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:10.917 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:10.917 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:10.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:10.931 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:10.931+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:11.031 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:11.032+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:11.035 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:11.035 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:11.035 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:11.035 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:11.035 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:11.044 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:11.132 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:11.133+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:11.167 INFO:teuthology.orchestra.run.smithi107.stdout:4788 write 0xa711000 thru 0xa71ca48 (0xba49 bytes) 2024-09-11T09:30:11.168 INFO:teuthology.orchestra.run.smithi107.stdout:4790 read 0x353fe00 thru 0x3548d7e (0x8f7f bytes) 2024-09-11T09:30:11.168 INFO:teuthology.orchestra.run.smithi107.stdout:4793 read 0x478c800 thru 0x47932a9 (0x6aaa bytes) 2024-09-11T09:30:11.181 INFO:teuthology.orchestra.run.smithi107.stdout:4795 write 0xb2c3800 thru 0xb2d0ac2 (0xd2c3 bytes) 2024-09-11T09:30:11.182 INFO:teuthology.orchestra.run.smithi107.stdout:4797 read 0x5cea600 thru 0x5cec0ec (0x1aed bytes) 2024-09-11T09:30:11.192 INFO:teuthology.orchestra.run.smithi107.stdout:4800 trunc from 0xb2d0c00 to 0x2aa9200 2024-09-11T09:30:11.201 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:11.201 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0103109529999998, 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.202 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:11.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:11.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0102902340000002, 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.207 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:11.211 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0102697159999998, 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.212 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.216 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0102470590000001, 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.221 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0102213369999999, 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:11.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:11.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0102012509999998, 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:11.226 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0101677250000001, 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:11.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:11.235 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0101447549999998, 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:11.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:11.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0101243649999998, 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.240 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:11.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0101043409999999, 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:11.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.01008473, 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:11.249 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:11.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:11.253 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0100596959999999, 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.254 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0100357789999999, 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34666671300000002, 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34664393500000001, 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.269 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:11.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34661632399999998, 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:11.274 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34659541900000002, 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:11.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:11.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34657507300000001, 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.285 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34639125199999998, 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.34636149700000002, 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:11.301 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:11.301 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:11.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:11.304 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:11.303+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4801 trunc from 0x2aa9200 to 0xc420800 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4804 read 0x2071400 thru 0x2080dbc (0xf9bd bytes) 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4806 trunc from 0xc420800 to 0x5347c00 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4807 write 0xd77ae00 thru 0xd78a594 (0xf795 bytes) 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4808 write 0xe969200 thru 0xe976570 (0xd371 bytes) 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4810 write 0xcbe800 thru 0xcc3ed9 (0x56da bytes) 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4811 read 0x2ea2400 thru 0x2ea76a4 (0x52a5 bytes) 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4812 read 0xce8ca00 thru 0xce9306e (0x666f bytes) 2024-09-11T09:30:11.305 INFO:teuthology.orchestra.run.smithi107.stdout:4814 clone 35 order 23 su 2097152 sc 10 2024-09-11T09:30:11.402 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:11.403+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:11.407 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:11.407 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:11.408 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:11.408 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:11.416 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:11.465 INFO:teuthology.orchestra.run.smithi017.stdout:osd.0: {} 2024-09-11T09:30:11.465 INFO:teuthology.orchestra.run.smithi017.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-11T09:30:11.482 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:11.482 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:11.498 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:11.498 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:11.503 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:11.504+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:11.515 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:11.515 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:11.531 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:11.532 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:11.548 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:11.548 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:11.560 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:11.560 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:11.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:11.604 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:11.605+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:11.670 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:11.670 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:11.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:11.706 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:11.706+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:11.784 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:11.784 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:11.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:11.806 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:11.807+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:11.900 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:11.900 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:11.907 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:30:11.907+0000 7f1dfc90a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32187) UID: 0 2024-09-11T09:30:11.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:11.957 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone34 from 0xe976600 (overlap 0x19f8600) to 0x298c00 2024-09-11T09:30:12.018 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.019 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.027 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:12.047 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:12.008+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:12.109 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:12.109+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:12.124 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-11T09:30:12.124 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:12.124 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-11T09:30:12.125 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T09:30:12.125 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:30:12.152 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.153 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.159 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T09:30:12.159 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.0 2024-09-11T09:30:12.160 INFO:tasks.thrashosds.thrasher:remote for osd 0 is ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:30:12.160 DEBUG:teuthology.orchestra.run.smithi017:> type ceph-objectstore-tool 2024-09-11T09:30:12.162 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:12.209 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:12.209+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:12.214 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-11T09:30:12.215 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op list-pgs 2024-09-11T09:30:12.263 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.263 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:12.312 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:12.313+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:12.411 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:12.411+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:12.415 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.415 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:12.512 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:12.512+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:12.532 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.532 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.541 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:12.625 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:12.626+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:12.690 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.690 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:12.712 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:12.713+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:12.813 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:12.814+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:12.847 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.847 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:12.867 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:30:12.943 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:12.944+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:12.975 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:12.975 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:12.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:13.015 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:13.015+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:13.116 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:13.117+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:13.147 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:13.148 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:13.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:13.217 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:13.217+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:13.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:13.357 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:13.357+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:13.420 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:13.420+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:13.427 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:13.427 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:13.437 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:13.520 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:13.521+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:13.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:13.583 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:13.583 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:13.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:13.633 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:13.633+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:13.671 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:13.722 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:13.722+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:13.735 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:13.735 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:13.744 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:13.823 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:13.824+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:13.858 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:30:13.888 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:13.889 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:13.892 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:13.892 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:13.901 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:13.906 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:13.906 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:13.922 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:13.938 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:13.939 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:13.946 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:13.947+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:13.955 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:13.955 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:13.958 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:13.959 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:14.007 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:14.007 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:14.007 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:14.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:14.039 INFO:teuthology.orchestra.run.smithi017.stdout:3.c 2024-09-11T09:30:14.039 INFO:teuthology.orchestra.run.smithi017.stdout:3.b 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.3 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.1f 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.1b 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.1 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.18 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.17 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.6 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.15 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.12 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:3.10 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:2.5 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:2.4 2024-09-11T09:30:14.040 INFO:teuthology.orchestra.run.smithi017.stdout:1.0 2024-09-11T09:30:14.057 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:14.058+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:14.126 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:14.127+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8238276369999999, 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8238044179999999, 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:14.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8237772569999997, 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:14.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:14.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8237567319999997, 2024-09-11T09:30:14.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:14.182 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8237302990000002, 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:14.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8237021469999997, 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8236717670000004, 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.193 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:14.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:14.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.823648693, 2024-09-11T09:30:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:14.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8236193319999998, 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.205 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:14.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.209 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8235981060000004, 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:14.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:14.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.823577813, 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:14.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8235517659999996, 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:14.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8235301850000001, 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.228 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8235093820000001, 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.233 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.823489972, 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:14.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8234530050000002, 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:14.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8234206029999998, 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:14.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:14.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:14.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8233984359999997, 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:14.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8233734559999997, 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:14.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:14.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 4.8233530770000002, 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:14.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.261 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.262 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:14.263 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-11T09:30:14.264 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:14.263+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:14.264 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:14.364 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:14.364+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:14.404 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:14.404 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:14.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:14.465 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:14.466+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:14.515 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:14.515 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:14.523 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:14.555 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --op export --pgid 3.3 --file /var/log/ceph/exp.3.3.0 2024-09-11T09:30:14.596 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:14.597+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.670190902999998, 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:14.654 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:14.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.670144858999997, 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:14.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.670133188999998, 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.664 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:14.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668844919999998, 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:14.668 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:14.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668809219000003, 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:14.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668787182000003, 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:14.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:14.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668751596999996, 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:14.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668691254000002, 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668642570000003, 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:14.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668586105999999, 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:14.696 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:14.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668566550999998, 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:14.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:14.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668518788, 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.706 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:14.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668405049999997, 2024-09-11T09:30:14.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:14.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668386306000002, 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:14.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:14.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668347316999998, 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:14.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:14.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:14.724 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 63.668324210000002, 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:14.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:14.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 44.960247719000002, 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:14.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:14.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.597848388999999, 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.735 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:14.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:14.740 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.592597311999999, 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.741 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 27.592336941999999, 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.746 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:14.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:14.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:14.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:14.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:14.752 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:14.753 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:14.753+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:14.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4815 read 0x8390200 thru 0x83983a9 (0x81aa bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4819 punch from 0x2ed8800 to 0x2edec00, (0x6400 bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4820 punch from 0x75a7400 to 0x75a8200, (0xe00 bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4821 read 0xa586c00 thru 0xa58a905 (0x3d06 bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4822 write 0x2af2e00 thru 0x2afffa1 (0xd1a2 bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4823 write 0x6f92400 thru 0x6f9798c (0x558d bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4825 punch from 0xe8c7800 to 0xe8ca400, (0x2c00 bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4827 punch from 0x78b9c00 to 0x78c5200, (0xb600 bytes) 2024-09-11T09:30:14.755 INFO:teuthology.orchestra.run.smithi107.stdout:4828 trunc from 0xe976600 to 0xaecb000 2024-09-11T09:30:14.778 INFO:teuthology.orchestra.run.smithi107.stdout:4829 write 0xd0ece00 thru 0xd0f3dba (0x6fbb bytes) 2024-09-11T09:30:14.779 INFO:teuthology.orchestra.run.smithi107.stdout:4830 read 0x3b1a000 thru 0x3b270c3 (0xd0c4 bytes) 2024-09-11T09:30:14.779 INFO:teuthology.orchestra.run.smithi107.stdout:4832 write 0x4317c00 thru 0x4322af1 (0xaef2 bytes) 2024-09-11T09:30:14.779 INFO:teuthology.orchestra.run.smithi107.stdout:4833 read 0xc26ac00 thru 0xc26c638 (0x1a39 bytes) 2024-09-11T09:30:14.779 INFO:teuthology.orchestra.run.smithi107.stdout:4835 write 0xb509a00 thru 0xb50e268 (0x4869 bytes) 2024-09-11T09:30:14.790 INFO:teuthology.orchestra.run.smithi107.stdout:4837 trunc from 0xd0f3e00 to 0x49c8800 2024-09-11T09:30:14.817 INFO:teuthology.orchestra.run.smithi107.stdout:4838 write 0x1b5600 thru 0x1b97b4 (0x41b5 bytes) 2024-09-11T09:30:14.822 INFO:teuthology.orchestra.run.smithi107.stdout:4840 punch from 0x14f3200 to 0x14fd400, (0xa200 bytes) 2024-09-11T09:30:14.823 INFO:teuthology.orchestra.run.smithi107.stdout:4843 flatten 2024-09-11T09:30:14.848 INFO:teuthology.orchestra.run.smithi107.stdout:4845 punch from 0x10b2400 to 0x10b7400, (0x5000 bytes) 2024-09-11T09:30:14.848 INFO:teuthology.orchestra.run.smithi107.stdout:4846 write 0x872a00 thru 0x876bf5 (0x41f6 bytes) 2024-09-11T09:30:14.848 INFO:teuthology.orchestra.run.smithi107.stdout:4848 read 0x1347a00 thru 0x1350854 (0x8e55 bytes) 2024-09-11T09:30:14.854 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:14.854+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:14.871 INFO:teuthology.orchestra.run.smithi107.stdout:4849 write 0xa744600 thru 0xa7489bf (0x43c0 bytes) 2024-09-11T09:30:14.872 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:14.872 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:14.872 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:14.872 INFO:teuthology.orchestra.run.smithi107.stdout:4850 read 0xea8a00 thru 0xeb3642 (0xac43 bytes) 2024-09-11T09:30:14.873 INFO:teuthology.orchestra.run.smithi107.stdout:4852 read 0x6bb9e00 thru 0x6bc4e89 (0xb08a bytes) 2024-09-11T09:30:14.873 INFO:teuthology.orchestra.run.smithi107.stdout:4854 write 0x11b3c00 thru 0x11c3085 (0xf486 bytes) 2024-09-11T09:30:14.874 INFO:teuthology.orchestra.run.smithi107.stdout:4856 write 0x29bc600 thru 0x29c3700 (0x7101 bytes) 2024-09-11T09:30:14.874 INFO:teuthology.orchestra.run.smithi107.stdout:4857 read 0x971a00 thru 0x97958c (0x7b8d bytes) 2024-09-11T09:30:14.875 INFO:teuthology.orchestra.run.smithi107.stdout:4863 read 0x68e6a00 thru 0x68ef07d (0x867e bytes) 2024-09-11T09:30:14.875 INFO:teuthology.orchestra.run.smithi107.stdout:4865 read 0x4e47200 thru 0x4e4c579 (0x537a bytes) 2024-09-11T09:30:14.875 INFO:teuthology.orchestra.run.smithi107.stdout:4866 read 0x3bf000 thru 0x3ce882 (0xf883 bytes) 2024-09-11T09:30:14.876 INFO:teuthology.orchestra.run.smithi107.stdout:4867 punch from 0x7bda400 to 0x7be2800, (0x8400 bytes) 2024-09-11T09:30:14.876 INFO:teuthology.orchestra.run.smithi107.stdout:4868 read 0x1503600 thru 0x150b9b0 (0x83b1 bytes) 2024-09-11T09:30:14.879 INFO:teuthology.orchestra.run.smithi107.stdout:4869 read 0x1cf1000 thru 0x1cf2f24 (0x1f25 bytes) 2024-09-11T09:30:14.879 INFO:teuthology.orchestra.run.smithi107.stdout:4870 read 0x6f08e00 thru 0x6f08f66 (0x167 bytes) 2024-09-11T09:30:14.881 INFO:teuthology.orchestra.run.smithi107.stdout:4874 trunc from 0xa748a00 to 0x8957800 2024-09-11T09:30:14.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:14.902 INFO:teuthology.orchestra.run.smithi107.stdout:4879 write 0x860c000 thru 0x8617fa2 (0xbfa3 bytes) 2024-09-11T09:30:14.902 INFO:teuthology.orchestra.run.smithi107.stdout:4885 write 0x5cc4200 thru 0x5cd106d (0xce6e bytes) 2024-09-11T09:30:14.904 INFO:teuthology.orchestra.run.smithi107.stdout:4886 trunc from 0x8957800 to 0xa37ac00 2024-09-11T09:30:14.917 INFO:teuthology.orchestra.run.smithi107.stdout:4889 write 0x1ace800 thru 0x1add029 (0xe82a bytes) 2024-09-11T09:30:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:4890 write 0x8171000 thru 0x817f93c (0xe93d bytes) 2024-09-11T09:30:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:4891 read 0x9865400 thru 0x9867ec3 (0x2ac4 bytes) 2024-09-11T09:30:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:4892 write 0x6e47e00 thru 0x6e52937 (0xab38 bytes) 2024-09-11T09:30:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:4893 read 0x7f5c600 thru 0x7f6594a (0x934b bytes) 2024-09-11T09:30:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:4894 punch from 0x46cf800 to 0x46d2000, (0x2800 bytes) 2024-09-11T09:30:14.924 INFO:teuthology.orchestra.run.smithi107.stdout:4895 punch from 0x1af3c00 to 0x1afe000, (0xa400 bytes) 2024-09-11T09:30:14.925 INFO:teuthology.orchestra.run.smithi107.stdout:4896 trunc from 0xa37ac00 to 0x8e62600 2024-09-11T09:30:14.941 INFO:teuthology.orchestra.run.smithi107.stdout:4898 read 0x3f54a00 thru 0x3f5d11f (0x8720 bytes) 2024-09-11T09:30:14.945 INFO:teuthology.orchestra.run.smithi107.stdout:4901 trunc from 0x8e62600 to 0xb332000 2024-09-11T09:30:14.954 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:14.955+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:14.968 INFO:teuthology.orchestra.run.smithi107.stdout:4902 write 0xe646800 thru 0xe64c67f (0x5e80 bytes) 2024-09-11T09:30:14.969 INFO:teuthology.orchestra.run.smithi107.stdout:4906 write 0x24e5e00 thru 0x24f159d (0xb79e bytes) 2024-09-11T09:30:14.985 INFO:teuthology.orchestra.run.smithi107.stdout:4907 trunc from 0xe64c800 to 0x1440800 2024-09-11T09:30:14.992 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:14.992 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:14.992 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:14.992 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:14.992 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:15.001 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:15.015 INFO:teuthology.orchestra.run.smithi107.stdout:4909 punch from 0x4e9c00 to 0x4f0200, (0x6600 bytes) 2024-09-11T09:30:15.017 INFO:teuthology.orchestra.run.smithi107.stdout:4910 trunc from 0x1440800 to 0x3c7000 2024-09-11T09:30:15.055 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:15.056+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:15.074 INFO:teuthology.orchestra.run.smithi107.stdout:4911 write 0x7620200 thru 0x762acea (0xaaeb bytes) 2024-09-11T09:30:15.074 INFO:teuthology.orchestra.run.smithi107.stdout:4914 clone 36 order 25 su 524288 sc 11 2024-09-11T09:30:15.112 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:15.112 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:15.112 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:15.112 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.201136155, 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.113 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:15.115 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.116 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.200905582999994, 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:15.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:15.120 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.121 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.200808378999994, 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.122 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.123 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.124 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.125 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.200573614999996, 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:15.127 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.131 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.200377599000007, 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.132 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.136 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.200109162000004, 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:15.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:15.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.141 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.200007412000005, 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:15.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.146 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.199613755000001, 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:15.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:15.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:15.151 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.199487509999997, 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:15.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:15.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:15.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.199278128000003, 2024-09-11T09:30:15.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:15.160 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.199167179, 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.161 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:15.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.198911241000005, 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:15.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.198758712, 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.170 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:15.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.174 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.198566348, 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.175 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:15.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.179 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.198235929999996, 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.180 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:15.184 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.197959600000004, 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.185 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:15.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:15.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:15.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.191 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.191 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:15.191 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.197905680000005, 2024-09-11T09:30:15.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:15.191 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.192 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.192 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.192 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.192 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.192 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:15.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.193 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:15.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.197705111000005, 2024-09-11T09:30:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.197349571999993, 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:15.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.418084039, 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:15.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:15.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.221 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:15.225 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:15.226 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:15.226+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:15.326 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:15.326+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:15.375 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:15.375 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:15.375 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:15.375 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:15.384 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:15.427 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:15.427+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:15.492 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:15.492 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:15.492 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:15.492 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:15.493 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:15.502 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:15.545 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:15.546+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:15.615 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:15.615 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4244350350000001, 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.616 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:15.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:15.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.424414316, 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.621 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4243937979999997, 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.626 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:15.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:15.630 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.424371141, 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:15.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.635 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4243454189999998, 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4243253329999996, 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.641 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:15.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4242918070000004, 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:15.645 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4242688369999996, 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:15.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:15.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:15.654 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4242484470000001, 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:15.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4242284229999997, 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.659 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4242088119999998, 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.668 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4241837779999997, 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:15.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 6.4241598609999997, 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.673 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.677 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.7607907950000001, 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:15.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:15.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.7607680170000002, 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:15.683 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:15.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.760740406, 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:15.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.690 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.7607195009999996, 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:15.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:15.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.700 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.7606991550000002, 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.701 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:15.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.7605153339999999, 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:15.706 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:15.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 4.760485579, 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:15.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:15.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:15.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:15.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:15.717 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:15.717 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:15.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:15.719 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:15.719+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:15.820 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:15.820+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:15.828 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:15.828 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:15.828 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:15.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:15.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:30:15.921 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:15.922+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:15.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:15.984 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:15.985 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:15.985 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:15.985 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:15.985 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:15.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:16.022 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:16.022+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:16.070 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:16.087 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:16.088 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:16.100 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:16.100 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:16.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:16.100 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:16.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.662856334, 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:16.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:16.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.046950606999999, 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:16.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:16.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:16.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.113 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.046752905999998, 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.114 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:16.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:16.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:16.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.04649448, 2024-09-11T09:30:16.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:16.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:16.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.126 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.046464596, 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:16.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045931444000001, 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:16.131 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:16.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045053995, 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:16.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:16.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:16.140 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.045015029000002, 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:16.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.044971553, 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:16.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.044846935999999, 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:16.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.044281466000001, 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:16.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:16.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.044250829999999, 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:16.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:16.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.042114397999999, 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:16.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:16.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.040593909999998, 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:16.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:16.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:16.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.036520080999999, 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:16.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.036406286999998, 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:16.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:16.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:16.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.184 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.036321605000001, 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:16.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:16.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.036275614000001, 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:16.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:16.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:16.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.035904316, 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:16.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 29.035762367, 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:16.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:16.205 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:16.206 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:16.207 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:16.207 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:16.208 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:16.208 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:16.209 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:16.207+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:16.209 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:16.209 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:16.209 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:16.210 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:16.210 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:16.210 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:16.258 INFO:teuthology.orchestra.run.smithi017.stderr:Exporting 3.3 info 3.3( v 87'528 (0'0,87'528] local-lis/les=21/22 n=56 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-11T09:30:16.269 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c063065c:::rbd_data.1106541f23ef.0000000000000000:11# 2024-09-11T09:30:16.269 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c063065c:::rbd_data.1106541f23ef.0000000000000000:head# 2024-09-11T09:30:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0788afc:::rbd_header.1106783b6c08:head# 2024-09-11T09:30:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0af3946:::rbd_data.110684ff6664.0000000000000083:head# 2024-09-11T09:30:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0ea1cec:::rbd_header.1106b949ffb0:head# 2024-09-11T09:30:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0f2b18c:::rbd_data.1106ebb745bf.00000000000000aa:d# 2024-09-11T09:30:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0f2b18c:::rbd_data.1106ebb745bf.00000000000000aa:head# 2024-09-11T09:30:16.270 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0f41ddd:::rbd_data.11064477c615.000000000000000f:21# 2024-09-11T09:30:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0f41ddd:::rbd_data.11064477c615.000000000000000f:head# 2024-09-11T09:30:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c0f587a8:::rbd_data.1106b643ff7f.00000000000000a0:head# 2024-09-11T09:30:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c1514339:::rbd_data.1106ebb745bf.0000000000000159:d# 2024-09-11T09:30:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c1514339:::rbd_data.1106ebb745bf.0000000000000159:head# 2024-09-11T09:30:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:head# 2024-09-11T09:30:16.272 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c21e1bd0:::rbd_data.1106b0e56eb1.0000000000000008:17# 2024-09-11T09:30:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c21e1bd0:::rbd_data.1106b0e56eb1.0000000000000008:head# 2024-09-11T09:30:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c23503b6:::rbd_data.110647be545c.0000000000000045:10# 2024-09-11T09:30:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c23503b6:::rbd_data.110647be545c.0000000000000045:head# 2024-09-11T09:30:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c252622e:::rbd_data.11062ef9d7e.000000000000001d:head# 2024-09-11T09:30:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c25b9571:::rbd_data.1106b643ff7f.00000000000000c2:head# 2024-09-11T09:30:16.281 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c268b360:::rbd_data.1106b949ffb0.0000000000000004:7# 2024-09-11T09:30:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c268b360:::rbd_data.1106b949ffb0.0000000000000004:head# 2024-09-11T09:30:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c2b8bd2b:::rbd_data.1106b643ff7f.000000000000009d:head# 2024-09-11T09:30:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c2c8a1e0:::rbd_data.1106b949ffb0.0000000000000007:7# 2024-09-11T09:30:16.282 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c2c8a1e0:::rbd_data.1106b949ffb0.0000000000000007:head# 2024-09-11T09:30:16.283 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c3073205:::rbd_data.1106aa24fca5.0000000000000007:22# 2024-09-11T09:30:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c3073205:::rbd_data.1106aa24fca5.0000000000000007:head# 2024-09-11T09:30:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c376c411:::rbd_data.1106b643ff7f.00000000000000df:head# 2024-09-11T09:30:16.285 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c391525a:::rbd_data.1106e0c0ccd2.0000000000000005:13# 2024-09-11T09:30:16.287 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c391525a:::rbd_data.1106e0c0ccd2.0000000000000005:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c3a5f7e2:::rbd_data.11062ef9d7e.0000000000000012:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c3eb6f47:::rbd_data.1106b643ff7f.0000000000000061:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c3f9d3e6:::rbd_data.1106ebb745bf.00000000000000c4:d# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c3f9d3e6:::rbd_data.1106ebb745bf.00000000000000c4:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c4274ace:::rbd_data.11066c82c820.0000000000000076:24# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c4274ace:::rbd_data.11066c82c820.0000000000000076:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c4330188:::rbd_data.1106b643ff7f.0000000000000046:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c4d83c86:::rbd_data.1106b643ff7f.0000000000000051:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c4e2a94f:::rbd_data.1106b643ff7f.000000000000009e:head# 2024-09-11T09:30:16.291 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c523c056:::rbd_data.11062ef9d7e.0000000000000035:head# 2024-09-11T09:30:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c5a67042:::rbd_data.1106783b6c08.0000000000000036:head# 2024-09-11T09:30:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c5acb606:::rbd_data.110647be545c.0000000000000025:10# 2024-09-11T09:30:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c5acb606:::rbd_data.110647be545c.0000000000000025:head# 2024-09-11T09:30:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6142745:::rbd_data.1106b643ff7f.000000000000000f:head# 2024-09-11T09:30:16.292 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6324a6f:::rbd_data.1106cda8c42d.000000000000000b:a# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6324a6f:::rbd_data.1106cda8c42d.000000000000000b:head# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c633986e:::rbd_data.1106783b6c08.0000000000000023:head# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c63515f1:::rbd_data.110647be545c.0000000000000044:10# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c63515f1:::rbd_data.110647be545c.0000000000000044:head# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c65cc037:::rbd_data.1106ebb745bf.00000000000000ac:d# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c65cc037:::rbd_data.1106ebb745bf.00000000000000ac:head# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6b16488:::rbd_data.11066fad748a.0000000000000030:head# 2024-09-11T09:30:16.293 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6d1c92f:::rbd_data.110660e40da5.000000000000001b:16# 2024-09-11T09:30:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6d1c92f:::rbd_data.110660e40da5.000000000000001b:head# 2024-09-11T09:30:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9# 2024-09-11T09:30:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:head# 2024-09-11T09:30:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:Read #3:c725f596:::rbd_data.11062c268ffd.0000000000000007:head# 2024-09-11T09:30:16.294 INFO:teuthology.orchestra.run.smithi017.stderr:Export successful 2024-09-11T09:30:16.308 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:16.308+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:16.342 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-11T09:30:16.357 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:16.357 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:16.366 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:16.409 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:16.409+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:16.413 INFO:teuthology.orchestra.run.smithi107.stdout:4915 punch from 0x3e600 to 0x4ce00, (0xe800 bytes) 2024-09-11T09:30:16.422 INFO:teuthology.orchestra.run.smithi107.stdout:4916 punch from 0x5d6fa00 to 0x5d7c000, (0xc600 bytes) 2024-09-11T09:30:16.440 INFO:teuthology.orchestra.run.smithi107.stdout:4918 write 0xb70dc00 thru 0xb7196c8 (0xbac9 bytes) 2024-09-11T09:30:16.447 INFO:teuthology.orchestra.run.smithi107.stdout:4919 trunc from 0xb719800 to 0xa3f1000 2024-09-11T09:30:16.478 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:16.478 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:16.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:16.492 INFO:teuthology.orchestra.run.smithi107.stdout:4920 write 0xe03ae00 thru 0xe03b5d6 (0x7d7 bytes) 2024-09-11T09:30:16.493 INFO:teuthology.orchestra.run.smithi107.stdout:4922 read 0x3099600 thru 0x30a2bab (0x95ac bytes) 2024-09-11T09:30:16.493 INFO:teuthology.orchestra.run.smithi107.stdout:4927 write 0x34f6c00 thru 0x34fc1c9 (0x55ca bytes) 2024-09-11T09:30:16.508 INFO:teuthology.orchestra.run.smithi107.stdout:4929 trunc from 0xe03b600 to 0x1ad1000 2024-09-11T09:30:16.531 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:16.532+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:16.533 INFO:teuthology.orchestra.run.smithi107.stdout:4930 read 0x226200 thru 0x228906 (0x2707 bytes) 2024-09-11T09:30:16.549 INFO:teuthology.orchestra.run.smithi107.stdout:4931 trunc from 0x1ad1000 to 0xdd34a00 2024-09-11T09:30:16.559 INFO:teuthology.orchestra.run.smithi107.stdout:4932 punch from 0x37be200 to 0x37bec00, (0xa00 bytes) 2024-09-11T09:30:16.574 INFO:teuthology.orchestra.run.smithi107.stdout:4934 trunc from 0xdd34a00 to 0x30c0c00 2024-09-11T09:30:16.598 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:16.598 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:16.607 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:16.621 INFO:teuthology.orchestra.run.smithi107.stdout:4936 write 0xe1d2400 thru 0xe1e0b3e (0xe73f bytes) 2024-09-11T09:30:16.639 INFO:teuthology.orchestra.run.smithi107.stdout:4937 trunc from 0xe1e0c00 to 0x57a8400 2024-09-11T09:30:16.648 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:16.649+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:16.671 INFO:teuthology.orchestra.run.smithi107.stdout:4940 write 0x2ee1c00 thru 0x2eead45 (0x9146 bytes) 2024-09-11T09:30:16.677 INFO:teuthology.orchestra.run.smithi107.stdout:4941 write 0x1036c00 thru 0x1040b58 (0x9f59 bytes) 2024-09-11T09:30:16.679 INFO:teuthology.orchestra.run.smithi107.stdout:4942 read 0x3a43800 thru 0x3a43c82 (0x483 bytes) 2024-09-11T09:30:16.701 INFO:teuthology.orchestra.run.smithi107.stdout:4943 write 0xc2dc000 thru 0xc2e105e (0x505f bytes) 2024-09-11T09:30:16.702 INFO:teuthology.orchestra.run.smithi107.stdout:4945 punch from 0x7cfec00 to 0x7d03a00, (0x4e00 bytes) 2024-09-11T09:30:16.707 INFO:teuthology.orchestra.run.smithi107.stdout:4946 write 0x249f400 thru 0x24ab015 (0xbc16 bytes) 2024-09-11T09:30:16.708 INFO:teuthology.orchestra.run.smithi107.stdout:4947 read 0x600cc00 thru 0x601a383 (0xd784 bytes) 2024-09-11T09:30:16.709 INFO:teuthology.orchestra.run.smithi107.stdout:4952 write 0x717000 thru 0x717dce (0xdcf bytes) 2024-09-11T09:30:16.710 INFO:teuthology.orchestra.run.smithi107.stdout:4955 read 0x30b5200 thru 0x30bb2ef (0x60f0 bytes) 2024-09-11T09:30:16.710 INFO:teuthology.orchestra.run.smithi107.stdout:4960 write 0x4f61a00 thru 0x4f6b96f (0x9f70 bytes) 2024-09-11T09:30:16.710 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:16.711+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:16.711 INFO:teuthology.orchestra.run.smithi107.stdout:4961 read 0x62bc00 thru 0x631db6 (0x61b7 bytes) 2024-09-11T09:30:16.711 INFO:teuthology.orchestra.run.smithi107.stdout:4962 read 0x6168400 thru 0x6172e30 (0xaa31 bytes) 2024-09-11T09:30:16.712 INFO:teuthology.orchestra.run.smithi107.stdout:4964 punch from 0x2796e00 to 0x27a2800, (0xba00 bytes) 2024-09-11T09:30:16.712 INFO:teuthology.orchestra.run.smithi107.stdout:4965 read 0x9e8ee00 thru 0x9e94e9f (0x60a0 bytes) 2024-09-11T09:30:16.723 INFO:teuthology.orchestra.run.smithi107.stdout:4966 write 0xe720400 thru 0xe7218ff (0x1500 bytes) 2024-09-11T09:30:16.728 INFO:teuthology.orchestra.run.smithi107.stdout:4971 write 0x8a87200 thru 0x8a8fc65 (0x8a66 bytes) 2024-09-11T09:30:16.729 INFO:teuthology.orchestra.run.smithi107.stdout:4973 read 0x2329a00 thru 0x233227d (0x887e bytes) 2024-09-11T09:30:16.730 INFO:teuthology.orchestra.run.smithi107.stdout:4975 trunc from 0xe721a00 to 0xdaa2200 2024-09-11T09:30:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:4977 read 0x2ff6400 thru 0x300273a (0xc33b bytes) 2024-09-11T09:30:16.754 INFO:teuthology.orchestra.run.smithi107.stdout:4978 write 0xc20b000 thru 0xc216927 (0xb928 bytes) 2024-09-11T09:30:16.754 INFO:teuthology.orchestra.run.smithi107.stdout:4983 punch from 0xca09400 to 0xca0f600, (0x6200 bytes) 2024-09-11T09:30:16.755 INFO:teuthology.orchestra.run.smithi107.stdout:4984 read 0x9b2800 thru 0x9bfbc8 (0xd3c9 bytes) 2024-09-11T09:30:16.761 INFO:teuthology.orchestra.run.smithi107.stdout:4985 trunc from 0xdaa2200 to 0xa5ee800 2024-09-11T09:30:16.763 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:16.764 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:16.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:16.789 INFO:teuthology.orchestra.run.smithi107.stdout:4989 write 0xc238e00 thru 0xc24409e (0xb29f bytes) 2024-09-11T09:30:16.790 INFO:teuthology.orchestra.run.smithi107.stdout:4992 read 0x82b3200 thru 0x82b4542 (0x1343 bytes) 2024-09-11T09:30:16.790 INFO:teuthology.orchestra.run.smithi107.stdout:4995 write 0x7ba0800 thru 0x7ba0c5a (0x45b bytes) 2024-09-11T09:30:16.791 INFO:teuthology.orchestra.run.smithi107.stdout:4996 write 0xb31e800 thru 0xb32c041 (0xd842 bytes) 2024-09-11T09:30:16.806 INFO:teuthology.orchestra.run.smithi107.stdout:4997 write 0xcd61800 thru 0xcd63c1a (0x241b bytes) 2024-09-11T09:30:16.815 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal --force --op remove --pgid 3.3 2024-09-11T09:30:16.816 INFO:teuthology.orchestra.run.smithi107.stdout:4998 trunc from 0xcd63e00 to 0xabdc400 2024-09-11T09:30:16.818 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:16.819+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:16.830 INFO:teuthology.orchestra.run.smithi107.stdout:4999 punch from 0x7bdc200 to 0x7be9e00, (0xdc00 bytes) 2024-09-11T09:30:16.838 INFO:teuthology.orchestra.run.smithi107.stdout:5002 write 0x7665400 thru 0x766acbb (0x58bc bytes) 2024-09-11T09:30:16.840 INFO:teuthology.orchestra.run.smithi107.stdout:5008 read 0x5bd1c00 thru 0x5bdc14c (0xa54d bytes) 2024-09-11T09:30:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:5011 trunc from 0xabdc400 to 0x4a1ae00 2024-09-11T09:30:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:30:16.887 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:16.888 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:16.892 INFO:teuthology.orchestra.run.smithi107.stdout:5012 write 0x96bbe00 thru 0x96bf467 (0x3668 bytes) 2024-09-11T09:30:16.893 INFO:teuthology.orchestra.run.smithi107.stdout:5013 read 0x28f0800 thru 0x28f6753 (0x5f54 bytes) 2024-09-11T09:30:16.893 INFO:teuthology.orchestra.run.smithi107.stdout:5015 write 0x153ce00 thru 0x1545c99 (0x8e9a bytes) 2024-09-11T09:30:16.895 INFO:teuthology.orchestra.run.smithi107.stdout:5017 read 0x35fec00 thru 0x3602e04 (0x4205 bytes) 2024-09-11T09:30:16.895 INFO:teuthology.orchestra.run.smithi107.stdout:5020 punch from 0x8e48a00 to 0x8e52600, (0x9c00 bytes) 2024-09-11T09:30:16.896 INFO:teuthology.orchestra.run.smithi107.stdout:5023 read 0x613ca00 thru 0x6148d56 (0xc357 bytes) 2024-09-11T09:30:16.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:16.907 INFO:teuthology.orchestra.run.smithi107.stdout:5024 trunc from 0x96bf600 to 0xc0200 2024-09-11T09:30:16.939 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:16.940+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:16.960 INFO:teuthology.orchestra.run.smithi107.stdout:5029 read 0x5c800 thru 0x6145a (0x4c5b bytes) 2024-09-11T09:30:16.989 INFO:teuthology.orchestra.run.smithi107.stdout:5030 write 0x9eda400 thru 0x9ee980a (0xf40b bytes) 2024-09-11T09:30:17.009 INFO:teuthology.orchestra.run.smithi107.stdout:5031 write 0xa5e0400 thru 0xa5eeebf (0xeac0 bytes) 2024-09-11T09:30:17.009 INFO:teuthology.orchestra.run.smithi107.stdout:5032 write 0x61b4200 thru 0x61bb9e7 (0x77e8 bytes) 2024-09-11T09:30:17.013 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:17.014+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:17.024 INFO:teuthology.orchestra.run.smithi107.stdout:5034 write 0xb977400 thru 0xb984ddc (0xd9dd bytes) 2024-09-11T09:30:17.025 INFO:teuthology.orchestra.run.smithi107.stdout:5036 punch from 0x1351a00 to 0x1361800, (0xfe00 bytes) 2024-09-11T09:30:17.027 INFO:teuthology.orchestra.run.smithi107.stdout:5037 write 0x967aa00 thru 0x9683b24 (0x9125 bytes) 2024-09-11T09:30:17.027 INFO:teuthology.orchestra.run.smithi107.stdout:5040 write 0xa57da00 thru 0xa584509 (0x6b0a bytes) 2024-09-11T09:30:17.030 INFO:teuthology.orchestra.run.smithi107.stdout:5042 read 0x90e9e00 thru 0x90ea6c4 (0x8c5 bytes) 2024-09-11T09:30:17.030 INFO:teuthology.orchestra.run.smithi107.stdout:5044 write 0xae2aa00 thru 0xae3143c (0x6a3d bytes) 2024-09-11T09:30:17.030 INFO:teuthology.orchestra.run.smithi107.stdout:5048 read 0xb6fe400 thru 0xb70c324 (0xdf25 bytes) 2024-09-11T09:30:17.030 INFO:teuthology.orchestra.run.smithi107.stdout:5049 write 0x13bc000 thru 0x13c5c12 (0x9c13 bytes) 2024-09-11T09:30:17.031 INFO:teuthology.orchestra.run.smithi107.stdout:5050 read 0x7bdfc00 thru 0x7be4fb3 (0x53b4 bytes) 2024-09-11T09:30:17.033 INFO:teuthology.orchestra.run.smithi107.stdout:5052 punch from 0xb2f4400 to 0xb2fb200, (0x6e00 bytes) 2024-09-11T09:30:17.033 INFO:teuthology.orchestra.run.smithi107.stdout:5055 write 0x10dc400 thru 0x10e0f3a (0x4b3b bytes) 2024-09-11T09:30:17.035 INFO:teuthology.orchestra.run.smithi107.stdout:5056 read 0x931200 thru 0x935fe5 (0x4de6 bytes) 2024-09-11T09:30:17.036 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:30:17.036 INFO:teuthology.orchestra.run.smithi107.stdout:5060 read 0x21f4600 thru 0x21f92d7 (0x4cd8 bytes) 2024-09-11T09:30:17.038 INFO:teuthology.orchestra.run.smithi107.stdout:5063 punch from 0x778aa00 to 0x7799c00, (0xf200 bytes) 2024-09-11T09:30:17.039 INFO:teuthology.orchestra.run.smithi107.stdout:5066 punch from 0x6117a00 to 0x611ae00, (0x3400 bytes) 2024-09-11T09:30:17.039 INFO:teuthology.orchestra.run.smithi107.stdout:5067 write 0x57a4c00 thru 0x57b15ce (0xc9cf bytes) 2024-09-11T09:30:17.046 INFO:teuthology.orchestra.run.smithi107.stdout:5069 trunc from 0xb984e00 to 0x5906e00 2024-09-11T09:30:17.055 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.055 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:17.072 INFO:teuthology.orchestra.run.smithi107.stdout:5070 read 0x36c4600 thru 0x36d158c (0xcf8d bytes) 2024-09-11T09:30:17.073 INFO:teuthology.orchestra.run.smithi107.stdout:5079 punch from 0x4188200 to 0x418bc00, (0x3a00 bytes) 2024-09-11T09:30:17.073 INFO:teuthology.orchestra.run.smithi107.stdout:5081 write 0x41f000 thru 0x41f880 (0x881 bytes) 2024-09-11T09:30:17.083 INFO:teuthology.orchestra.run.smithi107.stdout:5082 trunc from 0x5906e00 to 0xdad1000 2024-09-11T09:30:17.099 INFO:teuthology.orchestra.run.smithi107.stdout:5084 write 0x3b19a00 thru 0x3b2508f (0xb690 bytes) 2024-09-11T09:30:17.101 INFO:teuthology.orchestra.run.smithi107.stdout:5087 write 0x84f4c00 thru 0x8500319 (0xb71a bytes) 2024-09-11T09:30:17.101 INFO:teuthology.orchestra.run.smithi107.stdout:5088 read 0x6555800 thru 0x6557718 (0x1f19 bytes) 2024-09-11T09:30:17.111 INFO:teuthology.orchestra.run.smithi107.stdout:5090 trunc from 0xdad1000 to 0x5990000 2024-09-11T09:30:17.114 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:17.114+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:17.129 INFO:teuthology.orchestra.run.smithi107.stdout:5092 punch from 0x2555a00 to 0x255b000, (0x5600 bytes) 2024-09-11T09:30:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:5094 read 0x8f9a00 thru 0x9084ec (0xeaed bytes) 2024-09-11T09:30:17.138 INFO:teuthology.orchestra.run.smithi107.stdout:5095 read 0x3de2000 thru 0x3df0bff (0xec00 bytes) 2024-09-11T09:30:17.138 INFO:teuthology.orchestra.run.smithi107.stdout:5097 write 0x1233a00 thru 0x12439aa (0xffab bytes) 2024-09-11T09:30:17.154 INFO:teuthology.orchestra.run.smithi107.stdout:5098 write 0x9991800 thru 0x999ca6b (0xb26c bytes) 2024-09-11T09:30:17.156 INFO:teuthology.orchestra.run.smithi107.stdout:5102 punch from 0x47c9a00 to 0x47cda00, (0x4000 bytes) 2024-09-11T09:30:17.156 INFO:teuthology.orchestra.run.smithi107.stdout:5105 read 0x947b400 thru 0x947dbc6 (0x27c7 bytes) 2024-09-11T09:30:17.158 INFO:teuthology.orchestra.run.smithi107.stdout:5110 read 0x3d6d200 thru 0x3d71d3d (0x4b3e bytes) 2024-09-11T09:30:17.158 INFO:teuthology.orchestra.run.smithi107.stdout:5112 read 0x2066400 thru 0x206b189 (0x4d8a bytes) 2024-09-11T09:30:17.159 INFO:teuthology.orchestra.run.smithi107.stdout:5115 read 0x3afbe00 thru 0x3affef1 (0x40f2 bytes) 2024-09-11T09:30:17.159 INFO:teuthology.orchestra.run.smithi107.stdout:5119 read 0x4a61400 thru 0x4a65ea5 (0x4aa6 bytes) 2024-09-11T09:30:17.163 INFO:teuthology.orchestra.run.smithi107.stdout:5123 trunc from 0x999cc00 to 0xc213a00 2024-09-11T09:30:17.171 INFO:teuthology.orchestra.run.smithi107.stdout:5124 write 0xaaa0a00 thru 0xaaa40ae (0x36af bytes) 2024-09-11T09:30:17.179 INFO:teuthology.orchestra.run.smithi107.stdout:5127 read 0x4c46800 thru 0x4c4eec5 (0x86c6 bytes) 2024-09-11T09:30:17.179 INFO:teuthology.orchestra.run.smithi107.stdout:5128 punch from 0x2b70800 to 0x2b76e00, (0x6600 bytes) 2024-09-11T09:30:17.179 INFO:teuthology.orchestra.run.smithi107.stdout:5129 read 0x7f61200 thru 0x7f65be3 (0x49e4 bytes) 2024-09-11T09:30:17.182 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.182 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.190 INFO:teuthology.orchestra.run.smithi107.stdout:5131 write 0xdc70600 thru 0xdc7ab87 (0xa588 bytes) 2024-09-11T09:30:17.193 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:17.198 INFO:teuthology.orchestra.run.smithi107.stdout:5132 read 0xd9fc000 thru 0xda088f1 (0xc8f2 bytes) 2024-09-11T09:30:17.199 INFO:teuthology.orchestra.run.smithi107.stdout:5135 clone 37 order 20 su 32768 sc 8 2024-09-11T09:30:17.234 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:17.235+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:17.316 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:17.316+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:17.341 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.341 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:17.416 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:17.417+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:17.499 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.499 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.509 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:17.517 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:17.518+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:17.618 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:17.618+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:17.623 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.623 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.632 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:17.718 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:17.719+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:17.778 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.778 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.786 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:17.819 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:17.820+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:17.895 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:17.895 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:17.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:17.951 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:17.951+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:17.951 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone36 from 0xdc7ac00 (overlap 0xc0200) to 0x39a00 2024-09-11T09:30:18.022 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:18.022+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:18.022 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:18.022 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:18.031 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:18.122 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:18.122+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:18.175 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:18.175 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:18.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:18.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:18.255 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:18.256+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:18.287 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-11T09:30:18.289 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:18.292 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:18.298 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:18.323 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:18.341 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:18.341 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:18.342 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:18.343+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:18.360 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:18.360 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:18.377 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:18.377 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:18.395 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:18.395 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:18.406 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:18.406 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:18.406 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:18.414 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:18.414 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:18.416 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:18.417 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:18.417 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:18.456 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:18.457+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:18.518 INFO:teuthology.orchestra.run.smithi017.stdout: marking collection for removal 2024-09-11T09:30:18.518 INFO:teuthology.orchestra.run.smithi017.stdout:setting '_remove' omap key 2024-09-11T09:30:18.519 INFO:teuthology.orchestra.run.smithi017.stdout:finish_remove_pgs 3.3_head removing 3.3 2024-09-11T09:30:18.525 INFO:teuthology.orchestra.run.smithi017.stdout:Remove successful 2024-09-11T09:30:18.525 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:18.526+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:18.559 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:18.560 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:18.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:18.613 INFO:teuthology.orchestra.run.smithi107.stdout:5138 write 0x8f2ec00 thru 0x8f39376 (0xa777 bytes) 2024-09-11T09:30:18.614 INFO:teuthology.orchestra.run.smithi107.stdout:5140 write 0xd3ee200 thru 0xd3fd11b (0xef1c bytes) 2024-09-11T09:30:18.616 INFO:teuthology.orchestra.run.smithi107.stdout:5142 read 0xb9d8800 thru 0xb9d910a (0x90b bytes) 2024-09-11T09:30:18.617 INFO:teuthology.orchestra.run.smithi107.stdout:5144 read 0x4450c00 thru 0x4457a3a (0x6e3b bytes) 2024-09-11T09:30:18.618 INFO:teuthology.orchestra.run.smithi107.stdout:5147 write 0x5031600 thru 0x503d475 (0xbe76 bytes) 2024-09-11T09:30:18.620 INFO:teuthology.orchestra.run.smithi107.stdout:5148 punch from 0x293ce00 to 0x293fa00, (0x2c00 bytes) 2024-09-11T09:30:18.627 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:18.627+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:18.651 INFO:teuthology.orchestra.run.smithi107.stdout:5152 read 0xd72000 thru 0xd7fad7 (0xdad8 bytes) 2024-09-11T09:30:18.652 INFO:teuthology.orchestra.run.smithi107.stdout:5153 read 0x138200 thru 0x13f15d (0x6f5e bytes) 2024-09-11T09:30:18.652 INFO:teuthology.orchestra.run.smithi107.stdout:5157 read 0xa13de00 thru 0xa141bc0 (0x3dc1 bytes) 2024-09-11T09:30:18.653 INFO:teuthology.orchestra.run.smithi107.stdout:5158 read 0x5b77a00 thru 0x5b7d27b (0x587c bytes) 2024-09-11T09:30:18.653 INFO:teuthology.orchestra.run.smithi107.stdout:5159 write 0x8276800 thru 0x827f4b3 (0x8cb4 bytes) 2024-09-11T09:30:18.654 INFO:teuthology.orchestra.run.smithi107.stdout:5160 write 0xc875800 thru 0xc87bf94 (0x6795 bytes) 2024-09-11T09:30:18.654 INFO:teuthology.orchestra.run.smithi107.stdout:5162 punch from 0xbfe7c00 to 0xbfe9200, (0x1600 bytes) 2024-09-11T09:30:18.673 INFO:teuthology.orchestra.run.smithi107.stdout:5165 read 0x9c9cc00 thru 0x9ca0dd0 (0x41d1 bytes) 2024-09-11T09:30:18.674 INFO:teuthology.orchestra.run.smithi107.stdout:5166 write 0x255e000 thru 0x256191a (0x391b bytes) 2024-09-11T09:30:18.674 INFO:teuthology.orchestra.run.smithi107.stdout:5167 write 0x15c600 thru 0x164aa4 (0x84a5 bytes) 2024-09-11T09:30:18.675 INFO:teuthology.orchestra.run.smithi107.stdout:5169 punch from 0xaee7800 to 0xaef4c00, (0xd400 bytes) 2024-09-11T09:30:18.704 INFO:teuthology.orchestra.run.smithi107.stdout:5170 punch from 0x89e8e00 to 0x89eac00, (0x1e00 bytes) 2024-09-11T09:30:18.717 INFO:teuthology.orchestra.run.smithi107.stdout:5173 trunc from 0xdc7ac00 to 0x242d200 2024-09-11T09:30:18.727 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:18.728+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:18.746 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:18.746 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4062093680000007, 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:18.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:18.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4061861489999998, 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:18.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:18.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4061589879999996, 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:18.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:18.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:18.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4061384629999996, 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:18.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:18.764 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4061120299999992, 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:18.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:18.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4060838780000005, 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:18.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4060534980000003, 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:18.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:18.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:18.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4060304240000008, 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:18.778 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:18.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.782 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4060010629999997, 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:18.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4059798370000003, 2024-09-11T09:30:18.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:18.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:18.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4059595439999999, 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:18.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4059334969999995, 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:18.796 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.800 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4059119160000009, 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:18.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.805 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4058911129999991, 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:18.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4058717030000007, 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:18.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:18.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4058347359999992, 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:18.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4058023340000005, 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:18.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4057801669999996, 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:18.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4057551870000005, 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:18.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:18.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:18.833 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 9.4057348080000001, 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:18.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:18.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:18.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:18.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:18.838 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:18.838 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:18.839 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:18.839+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:18.840 INFO:teuthology.orchestra.run.smithi107.stdout:5174 punch from 0x156ee00 to 0x1575400, (0x6600 bytes) 2024-09-11T09:30:18.840 INFO:teuthology.orchestra.run.smithi107.stdout:5178 flatten 2024-09-11T09:30:18.840 INFO:teuthology.orchestra.run.smithi107.stdout:5183 punch from 0x204a00 to 0x20f800, (0xae00 bytes) 2024-09-11T09:30:18.840 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:5184 write 0xa7d5800 thru 0xa7dcdce (0x75cf bytes) 2024-09-11T09:30:18.855 INFO:teuthology.orchestra.run.smithi107.stdout:5185 write 0x36dca00 thru 0x36e6844 (0x9e45 bytes) 2024-09-11T09:30:18.855 INFO:teuthology.orchestra.run.smithi107.stdout:5191 write 0x6bb2600 thru 0x6bb6bd8 (0x45d9 bytes) 2024-09-11T09:30:18.855 INFO:teuthology.orchestra.run.smithi107.stdout:5193 write 0x7015400 thru 0x701f3cf (0x9fd0 bytes) 2024-09-11T09:30:18.855 INFO:teuthology.orchestra.run.smithi107.stdout:5194 write 0x2195000 thru 0x21960a6 (0x10a7 bytes) 2024-09-11T09:30:18.856 INFO:teuthology.orchestra.run.smithi107.stdout:5195 write 0x156dc00 thru 0x15763c9 (0x87ca bytes) 2024-09-11T09:30:18.856 INFO:teuthology.orchestra.run.smithi107.stdout:5196 write 0x24ee000 thru 0x24f8014 (0xa015 bytes) 2024-09-11T09:30:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:5197 punch from 0xa6bb200 to 0xa6c8800, (0xd600 bytes) 2024-09-11T09:30:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:5198 write 0xa261c00 thru 0xa26d161 (0xb562 bytes) 2024-09-11T09:30:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:5200 punch from 0x9e97400 to 0x9e98c00, (0x1800 bytes) 2024-09-11T09:30:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:5202 write 0x91e5a00 thru 0x91eedbf (0x93c0 bytes) 2024-09-11T09:30:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:5203 write 0x32ff800 thru 0x3308204 (0x8a05 bytes) 2024-09-11T09:30:18.860 INFO:teuthology.orchestra.run.smithi107.stdout:5204 write 0x3415c00 thru 0x341e59f (0x89a0 bytes) 2024-09-11T09:30:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:5205 write 0xaf44400 thru 0xaf481b2 (0x3db3 bytes) 2024-09-11T09:30:18.888 INFO:teuthology.orchestra.run.smithi107.stdout:5206 read 0x183c400 thru 0x184b2b1 (0xeeb2 bytes) 2024-09-11T09:30:18.892 INFO:teuthology.orchestra.run.smithi107.stdout:5207 trunc from 0xaf48200 to 0xe143600 2024-09-11T09:30:18.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:30:18.908 INFO:teuthology.orchestra.run.smithi107.stdout:5208 punch from 0x823f000 to 0x8243c00, (0x4c00 bytes) 2024-09-11T09:30:18.914 INFO:teuthology.orchestra.run.smithi107.stdout:5211 write 0xa9c4800 thru 0xa9cd098 (0x8899 bytes) 2024-09-11T09:30:18.914 INFO:teuthology.orchestra.run.smithi107.stdout:5215 write 0xded6c00 thru 0xdedc7b6 (0x5bb7 bytes) 2024-09-11T09:30:18.915 INFO:teuthology.orchestra.run.smithi107.stdout:5217 read 0xcf86000 thru 0xcf8b3e8 (0x53e9 bytes) 2024-09-11T09:30:18.915 INFO:teuthology.orchestra.run.smithi107.stdout:5218 read 0x90f1000 thru 0x90f6ae8 (0x5ae9 bytes) 2024-09-11T09:30:18.915 INFO:teuthology.orchestra.run.smithi107.stdout:5219 punch from 0xb3eb400 to 0xb3f1800, (0x6400 bytes) 2024-09-11T09:30:18.915 INFO:teuthology.orchestra.run.smithi107.stdout:5220 punch from 0x4a8200 to 0x4b1200, (0x9000 bytes) 2024-09-11T09:30:18.916 INFO:teuthology.orchestra.run.smithi107.stdout:5221 write 0x642a800 thru 0x6430782 (0x5f83 bytes) 2024-09-11T09:30:18.916 INFO:teuthology.orchestra.run.smithi107.stdout:5222 punch from 0x2d4fa00 to 0x2d51400, (0x1a00 bytes) 2024-09-11T09:30:18.916 INFO:teuthology.orchestra.run.smithi107.stdout:5226 read 0xf5ee00 thru 0xf6bc11 (0xce12 bytes) 2024-09-11T09:30:18.916 INFO:teuthology.orchestra.run.smithi107.stdout:5231 read 0x51b7400 thru 0x51c1fe0 (0xabe1 bytes) 2024-09-11T09:30:18.928 INFO:teuthology.orchestra.run.smithi107.stdout:5232 trunc from 0xe143600 to 0x3d44e00 2024-09-11T09:30:18.940 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:18.940+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:18.955 INFO:teuthology.orchestra.run.smithi107.stdout:5234 read 0x3ea400 thru 0x3f556e (0xb16f bytes) 2024-09-11T09:30:18.963 INFO:teuthology.orchestra.run.smithi107.stdout:5235 write 0x1ffde00 thru 0x200b500 (0xd701 bytes) 2024-09-11T09:30:18.977 INFO:teuthology.orchestra.run.smithi107.stdout:5238 write 0x55bd800 thru 0x55cb77b (0xdf7c bytes) 2024-09-11T09:30:18.978 INFO:teuthology.orchestra.run.smithi107.stdout:5242 clone 38 order 21 su 8192 sc 10 2024-09-11T09:30:19.004 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:19.004 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:19.004 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:19.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:19.033 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-0 --journal-path /var/lib/ceph/osd/ceph-0/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.3.0 2024-09-11T09:30:19.078 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:19.079+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:19.142 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:19.142+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:19.155 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:19.155 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:19.156 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:19.156 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:19.164 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:19.247 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:19.248+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.325443897, 2024-09-11T09:30:19.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:19.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:19.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.325397852999998, 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:19.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.325386183000006, 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:19.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:19.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:19.323 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.324097914000006, 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:19.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:19.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.324062213000005, 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:19.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.332 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.324040175999997, 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:19.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.324004591000005, 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:19.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323944248000004, 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:19.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:19.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323895563999997, 2024-09-11T09:30:19.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:19.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:19.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323839100000001, 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.351 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:19.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323819545000006, 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:19.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:19.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323771781999994, 2024-09-11T09:30:19.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:19.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.364 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323658043999998, 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:19.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:19.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.369 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323639299999996, 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:19.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323600311000007, 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:19.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.323577204000003, 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:19.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 49.615500713000003, 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:19.383 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:19.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.253101383000001, 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:19.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:19.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:19.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.247850305999997, 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:19.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:19.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:19.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.247589935999997, 2024-09-11T09:30:19.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:19.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:19.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:19.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:19.405 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:19.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:19.407 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:19.407+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:19.508 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:19.508+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:19.514 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:19.514 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:19.515 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:19.515 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:19.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:19.608 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:19.609+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:19.674 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:19.674 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:19.675 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:19.675 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:19.675 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:19.684 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:19.727 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:19.728+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.887313711999994, 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.799 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.887083140000001, 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:19.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.809 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.886985936000002, 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.886751172000004, 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.815 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:19.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:19.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.820 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.886555156, 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.821 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:19.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.886286718999997, 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:19.826 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:19.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.830 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.886184968999999, 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.885791311999995, 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.836 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.840 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.885665067000005, 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:19.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:19.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.845 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.885455684999997, 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.846 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.849 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.850 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.885344735999993, 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:19.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:19.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.885088797999998, 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.855 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:19.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:19.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.884936268999994, 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:19.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:19.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.884743904999993, 2024-09-11T09:30:19.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:19.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:19.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.884413487000003, 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:19.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:19.873 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.884137156999998, 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:19.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:19.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.884083236999999, 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:19.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.882 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.883882667999998, 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.883 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:19.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.883527129000001, 2024-09-11T09:30:19.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 37.104261596000001, 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:19.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:19.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:19.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:19.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:19.898 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:19.900 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:19.900 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:30:19.901 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:19.900+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:20.001 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:20.001+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:20.053 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:20.053 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:20.053 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:20.053 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:20.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:20.103 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:20.102+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:20.203 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:20.204+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:20.210 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:20.210 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:20.210 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:20.210 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:20.210 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:20.219 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:20.304 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:20.305+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.332 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:20.333 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141167503, 2024-09-11T09:30:20.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:20.333 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141146784, 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:20.340 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:20.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141126266000001, 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:20.345 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:20.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:20.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:20.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141103609, 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.350 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.352 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:20.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141077887, 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.355 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:20.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:20.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141057801000001, 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.360 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.361 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:20.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141024274999999, 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.365 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.369 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.141001305, 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:20.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:20.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.374 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.140980915, 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:20.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:20.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:20.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.140960891000001, 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:20.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:20.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:20.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.383 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.14094128, 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:20.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.140916246, 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.389 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:20.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:20.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:20.393 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 11.140892329, 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:20.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4775232630000001, 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.398 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:20.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:20.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.403 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4775004850000002, 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:20.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:20.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:20.409 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.477472874, 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:20.410 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:20.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:20.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:20.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4774519690000005, 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:20.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:20.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4774316229999993, 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:20.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:20.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:20.427 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4772478020000008, 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:20.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:20.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 9.4772180469999991, 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:20.433 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:20.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:20.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:20.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:20.438 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:20.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:20.440 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-11T09:30:20.440 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:20.440+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:20.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:20.442 INFO:teuthology.orchestra.run.smithi107.stdout:5244 write 0x6834a00 thru 0x6835a9b (0x109c bytes) 2024-09-11T09:30:20.444 INFO:teuthology.orchestra.run.smithi107.stdout:5245 trunc from 0x6835c00 to 0x9328400 2024-09-11T09:30:20.462 INFO:teuthology.orchestra.run.smithi107.stdout:5247 punch from 0x6698a00 to 0x66a3c00, (0xb200 bytes) 2024-09-11T09:30:20.478 INFO:teuthology.orchestra.run.smithi107.stdout:5251 write 0xc2cce00 thru 0xc2da2d7 (0xd4d8 bytes) 2024-09-11T09:30:20.479 INFO:teuthology.orchestra.run.smithi107.stdout:5252 trunc from 0xc2da400 to 0xbfdc000 2024-09-11T09:30:20.500 INFO:teuthology.orchestra.run.smithi107.stdout:5256 write 0x879d400 thru 0x87a93cb (0xbfcc bytes) 2024-09-11T09:30:20.501 INFO:teuthology.orchestra.run.smithi107.stdout:5258 write 0x612be00 thru 0x613bd4a (0xff4b bytes) 2024-09-11T09:30:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:5260 write 0xebdc000 thru 0xebdec74 (0x2c75 bytes) 2024-09-11T09:30:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:5261 write 0xb02c200 thru 0xb033516 (0x7317 bytes) 2024-09-11T09:30:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:5263 write 0x5add600 thru 0x5ade09a (0xa9b bytes) 2024-09-11T09:30:20.542 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:20.541+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:5266 punch from 0x2d28400 to 0x2d29000, (0xc00 bytes) 2024-09-11T09:30:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:5267 read 0x408e000 thru 0x4097b6c (0x9b6d bytes) 2024-09-11T09:30:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:5268 read 0x25e8400 thru 0x25eb9ea (0x35eb bytes) 2024-09-11T09:30:20.544 INFO:teuthology.orchestra.run.smithi107.stdout:5272 write 0x7451a00 thru 0x74581ff (0x6800 bytes) 2024-09-11T09:30:20.546 INFO:teuthology.orchestra.run.smithi107.stdout:5273 punch from 0x3dde200 to 0x3de8200, (0xa000 bytes) 2024-09-11T09:30:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:5275 read 0xa514c00 thru 0xa521be6 (0xcfe7 bytes) 2024-09-11T09:30:20.550 INFO:teuthology.orchestra.run.smithi107.stdout:5276 punch from 0x7727c00 to 0x7728e00, (0x1200 bytes) 2024-09-11T09:30:20.551 INFO:teuthology.orchestra.run.smithi107.stdout:5279 punch from 0xb95600 to 0xb9aa00, (0x5400 bytes) 2024-09-11T09:30:20.556 INFO:teuthology.orchestra.run.smithi107.stdout:5280 write 0x2cd5600 thru 0x2ce2a57 (0xd458 bytes) 2024-09-11T09:30:20.583 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1487 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556556+0000", 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022438606, 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022457444999999999, 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1487 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556556+0000", 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556556+0000", 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556557+0000", 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4099999999999999e-07 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556569+0000", 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2303999999999999e-05 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556578+0000", 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4989999999999994e-06 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556583+0000", 2024-09-11T09:30:20.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6840000000000001e-06 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1532 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557272+0000", 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021723419000000001, 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021804721999999999, 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1532 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557272+0000", 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557272+0000", 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557272+0000", 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557274+0000", 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8509999999999999e-06 2024-09-11T09:30:20.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557279+0000", 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6659999999999996e-06 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557284+0000", 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5229999999999997e-06 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.590 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1628 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557736+0000", 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021258879000000001, 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021407435999999998, 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.591 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1628 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557736+0000", 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557736+0000", 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557737+0000", 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8099999999999998e-07 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557738+0000", 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8309999999999999e-06 2024-09-11T09:30:20.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557744+0000", 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5550000000000002e-06 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557748+0000", 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2799999999999997e-06 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1488 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556591+0000", 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022404125, 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022592911, 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1488 2024-09-11T09:30:20.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556591+0000", 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556591+0000", 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556591+0000", 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3300000000000003e-07 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556594+0000", 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.193e-06 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556599+0000", 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6030000000000004e-06 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556604+0000", 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7469999999999996e-06 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1629 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557754+0000", 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021241156000000001, 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021480519999999999, 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1629 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557754+0000", 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557754+0000", 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557754+0000", 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8500000000000002e-07 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557756+0000", 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.95e-06 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557762+0000", 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9120000000000003e-06 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557767+0000", 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4700000000000004e-06 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1489 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556610+0000", 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022385075000000001, 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022678857, 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1489 2024-09-11T09:30:20.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556610+0000", 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556610+0000", 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556611+0000", 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556613+0000", 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.137e-06 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556619+0000", 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.161e-06 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556623+0000", 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6439999999999999e-06 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1630 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557773+0000", 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021222279, 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021568041999999999, 2024-09-11T09:30:20.605 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1630 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557773+0000", 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557773+0000", 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557773+0000", 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9900000000000001e-07 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557775+0000", 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9740000000000001e-06 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557781+0000", 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9229999999999999e-06 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557786+0000", 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7099999999999998e-06 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1490 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556629+0000", 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022366394000000001, 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022764930999999999, 2024-09-11T09:30:20.609 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1490 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556629+0000", 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556629+0000", 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556629+0000", 2024-09-11T09:30:20.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556631+0000", 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.79e-06 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556636+0000", 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5570000000000003e-06 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556641+0000", 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5290000000000002e-06 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1631 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557792+0000", 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021203132, 2024-09-11T09:30:20.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021656170999999998, 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1631 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557792+0000", 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557792+0000", 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557792+0000", 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.89e-07 2024-09-11T09:30:20.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557794+0000", 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9829999999999999e-06 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557800+0000", 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-06 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557805+0000", 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5569999999999997e-06 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1491 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556646+0000", 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022348815000000001, 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022857202, 2024-09-11T09:30:20.617 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1491 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556646+0000", 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556646+0000", 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556647+0000", 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3099999999999998e-07 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556657+0000", 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0641e-05 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556664+0000", 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5379999999999999e-06 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556669+0000", 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7380000000000002e-06 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1632 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557811+0000", 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021184227999999999, 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021745243000000001, 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1632 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557811+0000", 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557811+0000", 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557811+0000", 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557813+0000", 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0930000000000001e-06 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557820+0000", 2024-09-11T09:30:20.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2879999999999998e-06 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557825+0000", 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5720000000000002e-06 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1492 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556675+0000", 2024-09-11T09:30:20.624 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.02231961, 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022933735, 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1492 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556675+0000", 2024-09-11T09:30:20.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556675+0000", 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556676+0000", 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556678+0000", 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556685+0000", 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4500000000000001e-06 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556690+0000", 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1209999999999998e-06 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1633 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557832+0000", 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021162747999999999, 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021830901999999999, 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1633 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557832+0000", 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557832+0000", 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557833+0000", 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7400000000000003e-07 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557835+0000", 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3039999999999999e-06 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557842+0000", 2024-09-11T09:30:20.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1890000000000001e-06 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557847+0000", 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9409999999999998e-06 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1493 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556696+0000", 2024-09-11T09:30:20.631 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022299279000000002, 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023020527999999998, 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1493 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556696+0000", 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556696+0000", 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556696+0000", 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4499999999999997e-07 2024-09-11T09:30:20.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556698+0000", 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9889999999999999e-06 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556704+0000", 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9889999999999997e-06 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556709+0000", 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8790000000000002e-06 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1634 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557854+0000", 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021141229000000001, 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021903962999999999, 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1634 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557854+0000", 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557854+0000", 2024-09-11T09:30:20.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557854+0000", 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557857+0000", 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3020000000000002e-06 2024-09-11T09:30:20.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557868+0000", 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1821000000000001e-05 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557873+0000", 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0379999999999999e-06 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1717 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558561+0000", 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020433824999999999, 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021232488000000001, 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.639 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1717 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558561+0000", 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558561+0000", 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558562+0000", 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8100000000000003e-07 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558564+0000", 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.232e-06 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558570+0000", 2024-09-11T09:30:20.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9880000000000001e-06 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558575+0000", 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5680000000000001e-06 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1494 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.556715+0000", 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.022279785, 2024-09-11T09:30:20.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023120358000000001, 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1494 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556715+0000", 2024-09-11T09:30:20.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556715+0000", 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556716+0000", 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556718+0000", 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0080000000000001e-06 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556724+0000", 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0340000000000002e-06 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.556730+0000", 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1589999999999998e-06 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1577 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557450+0000", 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021545412999999999, 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022441099999999999, 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1577 2024-09-11T09:30:20.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557450+0000", 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557450+0000", 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557450+0000", 2024-09-11T09:30:20.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557452+0000", 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1739999999999999e-06 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557458+0000", 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8050000000000003e-06 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557462+0000", 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3429999999999998e-06 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1718 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558581+0000", 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020414254999999999, 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021366860000000001, 2024-09-11T09:30:20.649 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1718 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558581+0000", 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558581+0000", 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558581+0000", 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558584+0000", 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2069999999999998e-06 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558589+0000", 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8050000000000003e-06 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558594+0000", 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6639999999999997e-06 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1578 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557478+0000", 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021517154, 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022520276999999998, 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1578 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557478+0000", 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557478+0000", 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557478+0000", 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3799999999999998e-07 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557480+0000", 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557488+0000", 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4699999999999996e-06 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557491+0000", 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0939999999999999e-06 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1719 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558600+0000", 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020395073, 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021451811000000001, 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1719 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558600+0000", 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558600+0000", 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558600+0000", 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6699999999999999e-07 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558602+0000", 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0930000000000001e-06 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558608+0000", 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7350000000000001e-06 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558613+0000", 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7219999999999999e-06 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1579 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557496+0000", 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021499494000000001, 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022608421, 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1579 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557496+0000", 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557496+0000", 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557496+0000", 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557497+0000", 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.249e-06 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557500+0000", 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557502+0000", 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3089999999999998e-06 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1720 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558619+0000", 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020376124999999998, 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021535482000000002, 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1720 2024-09-11T09:30:20.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558619+0000", 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558619+0000", 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558619+0000", 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9000000000000002e-07 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558621+0000", 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1569999999999998e-06 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558627+0000", 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9179999999999999e-06 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558632+0000", 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0130000000000001e-06 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1580 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557506+0000", 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021489504999999999, 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022702987000000001, 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1580 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557506+0000", 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557506+0000", 2024-09-11T09:30:20.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557506+0000", 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0800000000000001e-07 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557507+0000", 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.096e-06 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557510+0000", 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9079999999999999e-06 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557512+0000", 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3309999999999998e-06 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1721 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558639+0000", 2024-09-11T09:30:20.670 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020356297999999998, 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021622077999999999, 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1721 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558639+0000", 2024-09-11T09:30:20.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558639+0000", 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558639+0000", 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2100000000000002e-07 2024-09-11T09:30:20.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558641+0000", 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558647+0000", 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.835e-06 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558652+0000", 2024-09-11T09:30:20.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299999999999998e-06 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1581 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557515+0000", 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021480102000000001, 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022798038999999999, 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1581 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557515+0000", 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557515+0000", 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557515+0000", 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1e-07 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557519+0000", 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0659999999999997e-06 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557522+0000", 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0139999999999999e-06 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557525+0000", 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.5100000000000001e-06 2024-09-11T09:30:20.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1722 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558658+0000", 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020337366999999999, 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021704403000000001, 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1722 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558658+0000", 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558658+0000", 2024-09-11T09:30:20.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558658+0000", 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.77e-07 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558664+0000", 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5940000000000001e-06 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558669+0000", 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6219999999999997e-06 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558674+0000", 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6349999999999997e-06 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1582 3.12 3.367f1b12 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557528+0000", 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021467374000000001, 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022887609, 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1582 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557528+0000", 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557528+0000", 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557528+0000", 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557529+0000", 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2139999999999999e-06 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557532+0000", 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9840000000000001e-06 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557535+0000", 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4159999999999998e-06 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1723 3.a 3.e47cba2a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558680+0000", 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020314985000000001, 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021786448999999999, 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1723 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558680+0000", 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558680+0000", 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558680+0000", 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558683+0000", 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1119999999999999e-06 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558688+0000", 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6799999999999998e-06 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558693+0000", 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4329999999999997e-06 2024-09-11T09:30:20.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1740 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.559395+0000", 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.019600123000000001, 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021125079000000001, 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.689 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1740 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559395+0000", 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559395+0000", 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559396+0000", 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6300000000000005e-07 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559398+0000", 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3070000000000001e-06 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559405+0000", 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1470000000000004e-06 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559411+0000", 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5419999999999996e-06 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1741 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.559418+0000", 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.019577303000000001, 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.02115529, 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1741 2024-09-11T09:30:20.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559418+0000", 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559418+0000", 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559418+0000", 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5299999999999999e-07 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559421+0000", 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2900000000000001e-06 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559427+0000", 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:20.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559431+0000", 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5850000000000001e-06 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1660 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558173+0000", 2024-09-11T09:30:20.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020822205, 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022451909999999999, 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1660 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558173+0000", 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558173+0000", 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558173+0000", 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8999999999999996e-07 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558176+0000", 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1550000000000001e-06 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558182+0000", 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1920000000000002e-06 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558187+0000", 2024-09-11T09:30:20.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.023e-06 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1520 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557037+0000", 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021958071999999999, 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023646952999999998, 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1520 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557037+0000", 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557037+0000", 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557037+0000", 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.08e-07 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557039+0000", 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9609999999999999e-06 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557045+0000", 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8649999999999998e-06 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557050+0000", 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4660000000000001e-06 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.703 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1661 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558194+0000", 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020801364999999999, 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022541489000000001, 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1661 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558194+0000", 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558194+0000", 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558194+0000", 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:20.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558197+0000", 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3539999999999998e-06 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558203+0000", 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0249999999999999e-06 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558207+0000", 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7999999999999998e-06 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1521 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557056+0000", 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021939337, 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023740275000000002, 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1521 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557056+0000", 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557056+0000", 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557056+0000", 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-07 2024-09-11T09:30:20.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557058+0000", 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0999999999999998e-06 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557064+0000", 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.784e-06 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557068+0000", 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4120000000000003e-06 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1662 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558214+0000", 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020780696000000001, 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022661596999999999, 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1662 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558214+0000", 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558214+0000", 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558215+0000", 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999999e-07 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558217+0000", 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.232e-06 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558223+0000", 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6869999999999999e-06 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558227+0000", 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3220000000000003e-06 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1744 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.559489+0000", 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.019505822999999999, 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021472677999999999, 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1744 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559489+0000", 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559489+0000", 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559490+0000", 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559492+0000", 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0899999999999999e-06 2024-09-11T09:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559497+0000", 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.75e-06 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559502+0000", 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6569999999999996e-06 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.718 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1522 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557075+0000", 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021920104999999999, 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023948113, 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1522 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557075+0000", 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557075+0000", 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557075+0000", 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4799999999999999e-07 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557077+0000", 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0229999999999999e-06 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557083+0000", 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7080000000000002e-06 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557088+0000", 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.722 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1663 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558233+0000", 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020761791000000002, 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022843615000000001, 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.723 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1663 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558233+0000", 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558233+0000", 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558234+0000", 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.725 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558236+0000", 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.114e-06 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558242+0000", 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1999999999999999e-06 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558247+0000", 2024-09-11T09:30:20.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5689999999999998e-06 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1745 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.559508+0000", 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.019486705, 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.021621715, 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.727 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1745 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559508+0000", 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559508+0000", 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559509+0000", 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559511+0000", 2024-09-11T09:30:20.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1380000000000001e-06 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559517+0000", 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8629999999999996e-06 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.559522+0000", 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1523 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557094+0000", 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021901137000000001, 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024088103, 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1523 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557094+0000", 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557094+0000", 2024-09-11T09:30:20.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557094+0000", 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2100000000000002e-07 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557096+0000", 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0829999999999998e-06 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557102+0000", 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6450000000000002e-06 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557107+0000", 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5940000000000004e-06 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1664 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558253+0000", 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020742058000000001, 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.022982430000000002, 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1664 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558253+0000", 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558253+0000", 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558254+0000", 2024-09-11T09:30:20.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5000000000000003e-07 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558266+0000", 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2058e-05 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558272+0000", 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7220000000000002e-06 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558277+0000", 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0560000000000004e-06 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1524 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557113+0000", 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021882173000000001, 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024161645999999998, 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1524 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557113+0000", 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557113+0000", 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557113+0000", 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.08e-07 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557115+0000", 2024-09-11T09:30:20.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557121+0000", 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9540000000000001e-06 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557126+0000", 2024-09-11T09:30:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6999999999999999e-06 2024-09-11T09:30:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1665 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558285+0000", 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020709887, 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023016368999999998, 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1665 2024-09-11T09:30:20.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558285+0000", 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558285+0000", 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558286+0000", 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558288+0000", 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2809999999999998e-06 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558296+0000", 2024-09-11T09:30:20.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8469999999999996e-06 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558301+0000", 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7110000000000003e-06 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.748 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1525 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557132+0000", 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021862665999999999, 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024196664, 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1525 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557132+0000", 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557132+0000", 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557133+0000", 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6199999999999999e-07 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557135+0000", 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9879999999999999e-06 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557141+0000", 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1909999999999997e-06 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557146+0000", 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2109999999999997e-06 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1621 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557662+0000", 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021332806999999999, 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023693498, 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1621 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557662+0000", 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557662+0000", 2024-09-11T09:30:20.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557662+0000", 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.11e-07 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557664+0000", 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.077e-06 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557666+0000", 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8629999999999999e-06 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557669+0000", 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.712e-06 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1666 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558307+0000", 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020688149999999999, 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023075120000000001, 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1666 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558307+0000", 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558307+0000", 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558307+0000", 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8699999999999995e-07 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558310+0000", 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1449999999999998e-06 2024-09-11T09:30:20.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558316+0000", 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9739999999999999e-06 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558320+0000", 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000001e-06 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1526 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557152+0000", 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021843358, 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024257308000000002, 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1526 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557152+0000", 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557152+0000", 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557152+0000", 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557154+0000", 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557160+0000", 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3570000000000003e-06 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557166+0000", 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0640000000000001e-06 2024-09-11T09:30:20.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1622 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557672+0000", 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021322856000000001, 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023763216, 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1622 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557672+0000", 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557672+0000", 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557672+0000", 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3999999999999998e-07 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557673+0000", 2024-09-11T09:30:20.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.048e-06 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557676+0000", 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.96e-06 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557679+0000", 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8890000000000002e-06 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.768 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1667 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558326+0000", 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020668954, 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023145972000000001, 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.769 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1667 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558326+0000", 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558326+0000", 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558326+0000", 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4499999999999998e-07 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558328+0000", 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9920000000000002e-06 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558334+0000", 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.908e-06 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558339+0000", 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7650000000000001e-06 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1527 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.772 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557172+0000", 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021823367999999999, 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024352054000000001, 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1527 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557172+0000", 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557172+0000", 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557172+0000", 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557174+0000", 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9910000000000001e-06 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557180+0000", 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9680000000000003e-06 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557185+0000", 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5079999999999999e-06 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.775 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1623 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557682+0000", 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021312777000000001, 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023894044999999999, 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1623 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557682+0000", 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557682+0000", 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557683+0000", 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6899999999999998e-07 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557684+0000", 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.124e-06 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557687+0000", 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8650000000000001e-06 2024-09-11T09:30:20.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557689+0000", 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3300000000000001e-06 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1668 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558345+0000", 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020649644000000002, 2024-09-11T09:30:20.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023289217000000001, 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1668 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558345+0000", 2024-09-11T09:30:20.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558345+0000", 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558346+0000", 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0600000000000001e-07 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558348+0000", 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.26e-06 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558353+0000", 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5040000000000004e-06 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558357+0000", 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3340000000000003e-06 2024-09-11T09:30:20.782 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1528 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557191+0000", 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021804361000000001, 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024488230999999999, 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.783 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1528 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557191+0000", 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557191+0000", 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557191+0000", 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1500000000000005e-07 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557203+0000", 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2255999999999999e-05 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557212+0000", 2024-09-11T09:30:20.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8210000000000008e-06 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557218+0000", 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3109999999999996e-06 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1624 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557692+0000", 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021303145999999998, 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024014929000000001, 2024-09-11T09:30:20.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1624 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557692+0000", 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557692+0000", 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557692+0000", 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557693+0000", 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.173e-06 2024-09-11T09:30:20.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557696+0000", 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7800000000000001e-06 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557699+0000", 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.481e-06 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1669 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558363+0000", 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020632220999999999, 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023370583, 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1669 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.790 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558363+0000", 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558363+0000", 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558363+0000", 2024-09-11T09:30:20.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9999999999999998e-07 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558365+0000", 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558371+0000", 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4870000000000002e-06 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558376+0000", 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1756 3.11 3:89f5d333:::rbd_data.1106de8e4559.0000000000000002:head [call rbd.sparse_copyup in=991800b,set-alloc-hint object_size 2097152 write_size 2097152,write 1218048~2560 in=2560b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.576364+0000", 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0026314490000000001, 2024-09-11T09:30:20.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0054027600000000004, 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "waiting for sub ops", 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1756 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.576364+0000", 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.576364+0000", 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.576365+0000", 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.319e-06 2024-09-11T09:30:20.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.577651+0000", 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0012857120000000001 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.577663+0000", 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.203e-05 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.577669+0000", 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.577709+0000", 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0380999999999998e-05 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.577821+0000", 2024-09-11T09:30:20.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000112493 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.578999+0000", 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.001177834 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1529 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.798 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557223+0000", 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021771685999999998, 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024587186, 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1529 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557223+0000", 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557223+0000", 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557224+0000", 2024-09-11T09:30:20.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557225+0000", 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.468e-06 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557228+0000", 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.207e-06 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557231+0000", 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.48e-06 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1625 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557702+0000", 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021293494999999999, 2024-09-11T09:30:20.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024143906, 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1625 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557702+0000", 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557702+0000", 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557702+0000", 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557703+0000", 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0839999999999999e-06 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557706+0000", 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.976e-06 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557708+0000", 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.429e-06 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.805 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1670 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558381+0000", 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020613592, 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023504122999999998, 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1670 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558381+0000", 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558381+0000", 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558382+0000", 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558384+0000", 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1119999999999999e-06 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558390+0000", 2024-09-11T09:30:20.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9109999999999998e-06 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558394+0000", 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5379999999999996e-06 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1530 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557235+0000", 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021760432, 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024679109000000001, 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1530 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557235+0000", 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557235+0000", 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557235+0000", 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557237+0000", 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.793e-06 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557242+0000", 2024-09-11T09:30:20.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5509999999999999e-06 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557247+0000", 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.408e-06 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.813 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1626 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557711+0000", 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021283837999999999, 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024229193, 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1626 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557711+0000", 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557711+0000", 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557712+0000", 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.03e-07 2024-09-11T09:30:20.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557713+0000", 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.124e-06 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557716+0000", 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.8830000000000002e-06 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557718+0000", 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3700000000000002e-06 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1671 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558400+0000", 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020594679000000001, 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023566508999999999, 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1671 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558400+0000", 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558400+0000", 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558401+0000", 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5000000000000003e-07 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558403+0000", 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9989999999999998e-06 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558410+0000", 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6479999999999997e-06 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558414+0000", 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8420000000000004e-06 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.820 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1531 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557253+0000", 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021742471999999999, 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024742051000000001, 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1531 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557253+0000", 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557253+0000", 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557253+0000", 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2399999999999999e-07 2024-09-11T09:30:20.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557255+0000", 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.057e-06 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557261+0000", 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1990000000000003e-06 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557266+0000", 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6859999999999997e-06 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1627 3.a 3.aebd0e4a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.557721+0000", 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.021274217000000002, 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.024324813000000001, 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1627 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557721+0000", 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557721+0000", 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557721+0000", 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0900000000000001e-07 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557726+0000", 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4619999999999998e-06 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557729+0000", 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1310000000000002e-06 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.557731+0000", 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.447e-06 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5850.0:1672 3.a 3.40cad24a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e95)", 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:20.558421+0000", 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.020574333, 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.023677969, 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "queued for pg", 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5850", 2024-09-11T09:30:20.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/236480132", 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1672 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558421+0000", 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558421+0000", 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:20.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558421+0000", 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2399999999999999e-07 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558423+0000", 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2800000000000002e-06 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558429+0000", 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8250000000000001e-06 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:20.558434+0000", 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4440000000000001e-06 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:30:20.831 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 66 2024-09-11T09:30:20.832 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:20.833 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:20.833 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:20.834 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:20.835 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:20.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:20.836 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:20.838 INFO:teuthology.orchestra.run.smithi107.stdout:5281 read 0x2f2c200 thru 0x2f2f990 (0x3791 bytes) 2024-09-11T09:30:20.838 INFO:teuthology.orchestra.run.smithi107.stdout:5283 read 0x9ac5a00 thru 0x9ad029e (0xa89f bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5284 punch from 0xa39400 to 0xa3e400, (0x5000 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5285 trunc from 0xebdee00 to 0x6b71800 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5288 trunc from 0x6b71800 to 0x918b000 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5289 write 0x7989800 thru 0x798b5a3 (0x1da4 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5290 write 0x44db000 thru 0x44e0b1c (0x5b1d bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5291 punch from 0x2f84400 to 0x2f8c000, (0x7c00 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5292 write 0xa053000 thru 0xa056c7f (0x3c80 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:skipping zero length punch hole 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5296 write 0x5f5d600 thru 0x5f68139 (0xab3a bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5297 punch from 0x1dd1000 to 0x1dd8800, (0x7800 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5299 punch from 0x9e03a00 to 0x9e13200, (0xf800 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5300 write 0x8ae8400 thru 0x8af829f (0xfea0 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5301 write 0x4be3a00 thru 0x4bec030 (0x8631 bytes) 2024-09-11T09:30:20.839 INFO:teuthology.orchestra.run.smithi107.stdout:5302 write 0x50d0800 thru 0x50d6201 (0x5a02 bytes) 2024-09-11T09:30:20.840 INFO:teuthology.orchestra.run.smithi107.stdout:5304 write 0x7d25400 thru 0x7d2acec (0x58ed bytes) 2024-09-11T09:30:20.840 INFO:teuthology.orchestra.run.smithi107.stdout:5305 read 0x20f1400 thru 0x20ffea1 (0xeaa2 bytes) 2024-09-11T09:30:20.840 INFO:teuthology.orchestra.run.smithi107.stdout:5306 write 0xa50600 thru 0xa563f3 (0x5df4 bytes) 2024-09-11T09:30:20.840 INFO:teuthology.orchestra.run.smithi107.stdout:5307 read 0x9dd7000 thru 0x9ddf10e (0x810f bytes) 2024-09-11T09:30:20.840 INFO:teuthology.orchestra.run.smithi107.stdout:5308 read 0x76ce200 thru 0x76d49dd (0x67de bytes) 2024-09-11T09:30:20.840 INFO:teuthology.orchestra.run.smithi107.stdout:5310 trunc from 0xa056e00 to 0x91e00 2024-09-11T09:30:20.841 INFO:teuthology.orchestra.run.smithi017.stdout:get_pg_num_history pg_num_history pg_num_history(e89 pg_nums {1={13=1},2={16=8},3={21=32}} deleted_pools ) 2024-09-11T09:30:20.841 INFO:teuthology.orchestra.run.smithi017.stdout:Importing pgid 3.3 2024-09-11T09:30:20.841 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c063065c:::rbd_data.1106541f23ef.0000000000000000:11# 2024-09-11T09:30:20.841 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c063065c:::rbd_data.1106541f23ef.0000000000000000:head# 2024-09-11T09:30:20.841 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 11=[]:{11=[11]} 2024-09-11T09:30:20.841 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0788afc:::rbd_header.1106783b6c08:head# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0af3946:::rbd_data.110684ff6664.0000000000000083:head# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0ea1cec:::rbd_header.1106b949ffb0:head# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0f2b18c:::rbd_data.1106ebb745bf.00000000000000aa:d# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0f2b18c:::rbd_data.1106ebb745bf.00000000000000aa:head# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:snapset d=[]:{d=[d]} 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0f41ddd:::rbd_data.11064477c615.000000000000000f:21# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0f41ddd:::rbd_data.11064477c615.000000000000000f:head# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 21=[]:{21=[21]} 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c0f587a8:::rbd_data.1106b643ff7f.00000000000000a0:head# 2024-09-11T09:30:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.843 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c1514339:::rbd_data.1106ebb745bf.0000000000000159:d# 2024-09-11T09:30:20.843 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:20.843 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c1514339:::rbd_data.1106ebb745bf.0000000000000159:head# 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:snapset d=[]:{d=[d]} 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c18e3a4e:::rbd_data.11062c268ffd.0000000000000001:head# 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c21e1bd0:::rbd_data.1106b0e56eb1.0000000000000008:17# 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c21e1bd0:::rbd_data.1106b0e56eb1.0000000000000008:head# 2024-09-11T09:30:20.846 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 17=[]:{17=[17]} 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c23503b6:::rbd_data.110647be545c.0000000000000045:10# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c23503b6:::rbd_data.110647be545c.0000000000000045:head# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 10=[]:{10=[10]} 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c252622e:::rbd_data.11062ef9d7e.000000000000001d:head# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 8=[]:{} 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c25b9571:::rbd_data.1106b643ff7f.00000000000000c2:head# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c268b360:::rbd_data.1106b949ffb0.0000000000000004:7# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c268b360:::rbd_data.1106b949ffb0.0000000000000004:head# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 7=[]:{7=[7]} 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c2b8bd2b:::rbd_data.1106b643ff7f.000000000000009d:head# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c2c8a1e0:::rbd_data.1106b949ffb0.0000000000000007:7# 2024-09-11T09:30:20.847 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c2c8a1e0:::rbd_data.1106b949ffb0.0000000000000007:head# 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 7=[]:{7=[7]} 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c3073205:::rbd_data.1106aa24fca5.0000000000000007:22# 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c3073205:::rbd_data.1106aa24fca5.0000000000000007:head# 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 22=[]:{22=[22]} 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c376c411:::rbd_data.1106b643ff7f.00000000000000df:head# 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c391525a:::rbd_data.1106e0c0ccd2.0000000000000005:13# 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c391525a:::rbd_data.1106e0c0ccd2.0000000000000005:head# 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 13=[]:{13=[13]} 2024-09-11T09:30:20.848 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:20.849 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:20.850 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c3a5f7e2:::rbd_data.11062ef9d7e.0000000000000012:head# 2024-09-11T09:30:20.850 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 8=[]:{} 2024-09-11T09:30:20.850 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c3eb6f47:::rbd_data.1106b643ff7f.0000000000000061:head# 2024-09-11T09:30:20.850 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.850 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c3f9d3e6:::rbd_data.1106ebb745bf.00000000000000c4:d# 2024-09-11T09:30:20.850 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c3f9d3e6:::rbd_data.1106ebb745bf.00000000000000c4:head# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:snapset d=[]:{d=[d]} 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c4274ace:::rbd_data.11066c82c820.0000000000000076:24# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c4274ace:::rbd_data.11066c82c820.0000000000000076:head# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 24=[]:{24=[24]} 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c4330188:::rbd_data.1106b643ff7f.0000000000000046:head# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c4d83c86:::rbd_data.1106b643ff7f.0000000000000051:head# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c4e2a94f:::rbd_data.1106b643ff7f.000000000000009e:head# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c523c056:::rbd_data.11062ef9d7e.0000000000000035:head# 2024-09-11T09:30:20.851 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 8=[]:{} 2024-09-11T09:30:20.852 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:20.852 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:20.855 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c5a67042:::rbd_data.1106783b6c08.0000000000000036:head# 2024-09-11T09:30:20.855 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.855 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c5acb606:::rbd_data.110647be545c.0000000000000025:10# 2024-09-11T09:30:20.855 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c5acb606:::rbd_data.110647be545c.0000000000000025:head# 2024-09-11T09:30:20.855 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 10=[]:{10=[10]} 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6142745:::rbd_data.1106b643ff7f.000000000000000f:head# 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 1a=[]:{} 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6324a6f:::rbd_data.1106cda8c42d.000000000000000b:a# 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6324a6f:::rbd_data.1106cda8c42d.000000000000000b:head# 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:snapset a=[]:{a=[a]} 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c633986e:::rbd_data.1106783b6c08.0000000000000023:head# 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c63515f1:::rbd_data.110647be545c.0000000000000044:10# 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c63515f1:::rbd_data.110647be545c.0000000000000044:head# 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 10=[]:{10=[10]} 2024-09-11T09:30:20.856 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c65cc037:::rbd_data.1106ebb745bf.00000000000000ac:d# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c65cc037:::rbd_data.1106ebb745bf.00000000000000ac:head# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:snapset d=[]:{d=[d]} 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6b16488:::rbd_data.11066fad748a.0000000000000030:head# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:snapset c=[]:{} 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6d1c92f:::rbd_data.110660e40da5.000000000000001b:16# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6d1c92f:::rbd_data.110660e40da5.000000000000001b:head# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 16=[]:{16=[16]} 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:9# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c6d77a25:::rbd_data.1106259a042c.0000000000000010:head# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 9=[]:{9=[9]} 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:Write #3:c725f596:::rbd_data.11062c268ffd.0000000000000007:head# 2024-09-11T09:30:20.857 INFO:teuthology.orchestra.run.smithi017.stdout:snapset 0=[]:{} 2024-09-11T09:30:20.858 INFO:teuthology.orchestra.run.smithi017.stdout:write_pg epoch 87 info 3.3( v 87'528 (0'0,87'528] local-lis/les=21/22 n=56 ec=21/21 lis/c=21/21 les/c/f=22/22/0 sis=21) 2024-09-11T09:30:20.858 INFO:teuthology.orchestra.run.smithi017.stdout:Import successful 2024-09-11T09:30:20.858 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:20.833+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:20.866 INFO:teuthology.orchestra.run.smithi107.stdout:5312 trunc from 0x91e00 to 0x4149000 2024-09-11T09:30:20.875 INFO:teuthology.orchestra.run.smithi107.stdout:5313 read 0x3ece00 thru 0x3f0611 (0x3812 bytes) 2024-09-11T09:30:20.876 INFO:teuthology.orchestra.run.smithi107.stdout:5314 read 0x25a6400 thru 0x25a7cb7 (0x18b8 bytes) 2024-09-11T09:30:20.876 INFO:teuthology.orchestra.run.smithi107.stdout:5317 punch from 0x23b2400 to 0x23c0400, (0xe000 bytes) 2024-09-11T09:30:20.876 INFO:teuthology.orchestra.run.smithi107.stdout:5318 read 0x614400 thru 0x621633 (0xd234 bytes) 2024-09-11T09:30:20.897 INFO:teuthology.orchestra.run.smithi107.stdout:5321 write 0x9665a00 thru 0x9667ff6 (0x25f7 bytes) 2024-09-11T09:30:20.897 INFO:teuthology.orchestra.run.smithi107.stdout:5322 read 0x2bd0800 thru 0x2bdc7d4 (0xbfd5 bytes) 2024-09-11T09:30:20.898 INFO:teuthology.orchestra.run.smithi107.stdout:5325 write 0x21c6200 thru 0x21c637a (0x17b bytes) 2024-09-11T09:30:20.898 INFO:teuthology.orchestra.run.smithi107.stdout:5329 read 0x40d8000 thru 0x40e3f54 (0xbf55 bytes) 2024-09-11T09:30:20.898 INFO:teuthology.orchestra.run.smithi107.stdout:5330 write 0x2889600 thru 0x288a9e4 (0x13e5 bytes) 2024-09-11T09:30:20.898 INFO:teuthology.orchestra.run.smithi107.stdout:5331 read 0x17fb600 thru 0x1801e7c (0x687d bytes) 2024-09-11T09:30:20.902 INFO:teuthology.orchestra.run.smithi107.stdout:5333 read 0x2f86a00 thru 0x2f932c0 (0xc8c1 bytes) 2024-09-11T09:30:20.902 INFO:teuthology.orchestra.run.smithi107.stdout:5334 read 0x20bee00 thru 0x20c8a78 (0x9c79 bytes) 2024-09-11T09:30:20.921 INFO:teuthology.orchestra.run.smithi107.stdout:5335 write 0xd552a00 thru 0xd5627bc (0xfdbd bytes) 2024-09-11T09:30:20.928 INFO:teuthology.orchestra.run.smithi107.stdout:5336 read 0x30e1800 thru 0x30ecf8f (0xb790 bytes) 2024-09-11T09:30:20.929 INFO:teuthology.orchestra.run.smithi107.stdout:5340 punch from 0x59ab600 to 0x59ae400, (0x2e00 bytes) 2024-09-11T09:30:20.929 INFO:teuthology.orchestra.run.smithi107.stdout:5343 punch from 0xa9d1800 to 0xa9dca00, (0xb200 bytes) 2024-09-11T09:30:20.929 INFO:teuthology.orchestra.run.smithi107.stdout:5344 punch from 0x6bff200 to 0x6c0c000, (0xce00 bytes) 2024-09-11T09:30:20.930 INFO:teuthology.orchestra.run.smithi107.stdout:5346 trunc from 0xd562800 to 0xd4ca400 2024-09-11T09:30:20.934 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:20.934+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:20.967 INFO:teuthology.orchestra.run.smithi107.stdout:5347 write 0x9562800 thru 0x9566b0a (0x430b bytes) 2024-09-11T09:30:20.975 INFO:teuthology.orchestra.run.smithi107.stdout:5349 trunc from 0xd4ca400 to 0xccf6c00 2024-09-11T09:30:20.996 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:20.996 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:20.997 INFO:teuthology.orchestra.run.smithi107.stdout:5350 read 0x5374600 thru 0x5375c3a (0x163b bytes) 2024-09-11T09:30:21.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:21.008 INFO:teuthology.orchestra.run.smithi107.stdout:5352 trunc from 0xccf6c00 to 0x34c3600 2024-09-11T09:30:21.038 INFO:teuthology.orchestra.run.smithi107.stdout:5356 read 0x2573c00 thru 0x2576b81 (0x2f82 bytes) 2024-09-11T09:30:21.039 INFO:teuthology.orchestra.run.smithi107.stdout:5357 write 0x216400 thru 0x21d96b (0x756c bytes) 2024-09-11T09:30:21.039 INFO:teuthology.orchestra.run.smithi107.stdout:5358 write 0x788400 thru 0x79662e (0xe22f bytes) 2024-09-11T09:30:21.040 INFO:teuthology.orchestra.run.smithi107.stdout:5359 read 0x11a5000 thru 0x11b085f (0xb860 bytes) 2024-09-11T09:30:21.041 INFO:teuthology.orchestra.run.smithi107.stdout:5360 read 0x236000 thru 0x244856 (0xe857 bytes) 2024-09-11T09:30:21.046 INFO:teuthology.orchestra.run.smithi107.stdout:5361 trunc from 0x34c3600 to 0x7328800 2024-09-11T09:30:21.047 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:21.048+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:21.063 INFO:teuthology.orchestra.run.smithi107.stdout:5363 write 0x5f5c600 thru 0x5f605a2 (0x3fa3 bytes) 2024-09-11T09:30:21.064 INFO:teuthology.orchestra.run.smithi107.stdout:5367 punch from 0x13cea00 to 0x13d2c00, (0x4200 bytes) 2024-09-11T09:30:21.065 INFO:teuthology.orchestra.run.smithi107.stdout:5368 read 0x1417800 thru 0x1423200 (0xba01 bytes) 2024-09-11T09:30:21.069 INFO:teuthology.orchestra.run.smithi107.stdout:5369 read 0x3d0fc00 thru 0x3d1675a (0x6b5b bytes) 2024-09-11T09:30:21.069 INFO:teuthology.orchestra.run.smithi107.stdout:5370 punch from 0x1649400 to 0x1653a00, (0xa600 bytes) 2024-09-11T09:30:21.070 INFO:teuthology.orchestra.run.smithi107.stdout:5373 trunc from 0x7328800 to 0x661c800 2024-09-11T09:30:21.085 INFO:teuthology.orchestra.run.smithi107.stdout:5374 punch from 0x28fa000 to 0x28fa600, (0x600 bytes) 2024-09-11T09:30:21.086 INFO:teuthology.orchestra.run.smithi107.stdout:5375 read 0x3dc3c00 thru 0x3dd06a1 (0xcaa2 bytes) 2024-09-11T09:30:21.087 INFO:teuthology.orchestra.run.smithi107.stdout:5376 read 0x3bd8c00 thru 0x3bdc4e5 (0x38e6 bytes) 2024-09-11T09:30:21.088 INFO:teuthology.orchestra.run.smithi107.stdout:5380 trunc from 0x661c800 to 0x4b7ba00 2024-09-11T09:30:21.102 INFO:teuthology.orchestra.run.smithi107.stdout:5381 read 0x1721a00 thru 0x172703b (0x563c bytes) 2024-09-11T09:30:21.109 INFO:teuthology.orchestra.run.smithi107.stdout:5382 punch from 0x4666e00 to 0x466f600, (0x8800 bytes) 2024-09-11T09:30:21.113 INFO:teuthology.orchestra.run.smithi107.stdout:5384 trunc from 0x4b7ba00 to 0x7e5a000 2024-09-11T09:30:21.132 INFO:teuthology.orchestra.run.smithi107.stdout:5386 trunc from 0x7e5a000 to 0xc709400 2024-09-11T09:30:21.135 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:21.135+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:21.144 INFO:teuthology.orchestra.run.smithi107.stdout:5387 trunc from 0xc709400 to 0x8d51800 2024-09-11T09:30:21.154 INFO:teuthology.orchestra.run.smithi107.stdout:5388 write 0x7e8f000 thru 0x7e9bec4 (0xcec5 bytes) 2024-09-11T09:30:21.158 INFO:teuthology.orchestra.run.smithi107.stdout:5390 trunc from 0x8d51800 to 0xb99c600 2024-09-11T09:30:21.166 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:21.166 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:21.176 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:21.180 INFO:teuthology.orchestra.run.smithi107.stdout:5392 punch from 0xa9d7600 to 0xa9df200, (0x7c00 bytes) 2024-09-11T09:30:21.182 INFO:teuthology.orchestra.run.smithi107.stdout:5394 read 0x623ee00 thru 0x6247b9e (0x8d9f bytes) 2024-09-11T09:30:21.182 INFO:teuthology.orchestra.run.smithi107.stdout:5396 read 0xde2e00 thru 0xde8ada (0x5cdb bytes) 2024-09-11T09:30:21.182 INFO:teuthology.orchestra.run.smithi107.stdout:5398 read 0x887ea00 thru 0x8887ffd (0x95fe bytes) 2024-09-11T09:30:21.183 INFO:teuthology.orchestra.run.smithi107.stdout:5400 punch from 0x54bcc00 to 0x54cc000, (0xf400 bytes) 2024-09-11T09:30:21.183 INFO:teuthology.orchestra.run.smithi107.stdout:5402 read 0x778e600 thru 0x778e7cc (0x1cd bytes) 2024-09-11T09:30:21.184 INFO:teuthology.orchestra.run.smithi107.stdout:5404 punch from 0x174c000 to 0x1753600, (0x7600 bytes) 2024-09-11T09:30:21.184 INFO:teuthology.orchestra.run.smithi107.stdout:5405 read 0x6bc400 thru 0x6ca399 (0xdf9a bytes) 2024-09-11T09:30:21.193 INFO:teuthology.orchestra.run.smithi107.stdout:5408 read 0x214e00 thru 0x22002a (0xb22b bytes) 2024-09-11T09:30:21.195 INFO:teuthology.orchestra.run.smithi107.stdout:5414 read 0xb836e00 thru 0xb84497d (0xdb7e bytes) 2024-09-11T09:30:21.206 INFO:teuthology.orchestra.run.smithi107.stdout:5415 write 0xc3a5e00 thru 0xc3b0d95 (0xaf96 bytes) 2024-09-11T09:30:21.207 INFO:teuthology.orchestra.run.smithi107.stdout:5419 write 0x3160200 thru 0x31607fa (0x5fb bytes) 2024-09-11T09:30:21.207 INFO:teuthology.orchestra.run.smithi107.stdout:5420 write 0x20a3c00 thru 0x20b23e8 (0xe7e9 bytes) 2024-09-11T09:30:21.208 INFO:teuthology.orchestra.run.smithi107.stdout:5423 punch from 0x5f2d600 to 0x5f36e00, (0x9800 bytes) 2024-09-11T09:30:21.209 INFO:teuthology.orchestra.run.smithi107.stdout:5424 trunc from 0xc3b0e00 to 0xbc8be00 2024-09-11T09:30:21.236 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:21.236+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:21.237 INFO:teuthology.orchestra.run.smithi107.stdout:5426 read 0x68da000 thru 0x68e252e (0x852f bytes) 2024-09-11T09:30:21.238 INFO:teuthology.orchestra.run.smithi107.stdout:5427 punch from 0xee9c00 to 0xef0e00, (0x7200 bytes) 2024-09-11T09:30:21.239 INFO:teuthology.orchestra.run.smithi107.stdout:5428 write 0x1a8be00 thru 0x1a969b9 (0xabba bytes) 2024-09-11T09:30:21.240 INFO:teuthology.orchestra.run.smithi107.stdout:5429 write 0x232f400 thru 0x23317a5 (0x23a6 bytes) 2024-09-11T09:30:21.241 INFO:teuthology.orchestra.run.smithi107.stdout:5430 punch from 0xb85fe00 to 0xb866400, (0x6600 bytes) 2024-09-11T09:30:21.241 INFO:teuthology.orchestra.run.smithi107.stdout:5433 write 0x28e9400 thru 0x28f1577 (0x8178 bytes) 2024-09-11T09:30:21.242 INFO:teuthology.orchestra.run.smithi107.stdout:5434 punch from 0xafc0e00 to 0xafc8c00, (0x7e00 bytes) 2024-09-11T09:30:21.242 INFO:teuthology.orchestra.run.smithi107.stdout:5435 write 0x4101e00 thru 0x410c190 (0xa391 bytes) 2024-09-11T09:30:21.243 INFO:teuthology.orchestra.run.smithi107.stdout:5438 punch from 0xb8f9400 to 0xb906e00, (0xda00 bytes) 2024-09-11T09:30:21.243 INFO:teuthology.orchestra.run.smithi107.stdout:5439 write 0x7451e00 thru 0x7456dc2 (0x4fc3 bytes) 2024-09-11T09:30:21.243 INFO:teuthology.orchestra.run.smithi107.stdout:5440 write 0x2073a00 thru 0x20800a7 (0xc6a8 bytes) 2024-09-11T09:30:21.251 INFO:teuthology.orchestra.run.smithi107.stdout:5442 write 0x378c600 thru 0x3797b9b (0xb59c bytes) 2024-09-11T09:30:21.251 INFO:teuthology.orchestra.run.smithi107.stdout:5444 punch from 0xb8bec00 to 0xb8c1800, (0x2c00 bytes) 2024-09-11T09:30:21.252 INFO:teuthology.orchestra.run.smithi107.stdout:5447 read 0x404e00 thru 0x40c52f (0x7730 bytes) 2024-09-11T09:30:21.254 INFO:teuthology.orchestra.run.smithi107.stdout:5448 read 0x3d3fa00 thru 0x3d44778 (0x4d79 bytes) 2024-09-11T09:30:21.260 INFO:teuthology.orchestra.run.smithi107.stdout:5449 read 0xa736800 thru 0xa736a70 (0x271 bytes) 2024-09-11T09:30:21.261 INFO:teuthology.orchestra.run.smithi107.stdout:5450 write 0x1b60400 thru 0x1b6e450 (0xe051 bytes) 2024-09-11T09:30:21.263 INFO:teuthology.orchestra.run.smithi107.stdout:5451 write 0x3325000 thru 0x332d23c (0x823d bytes) 2024-09-11T09:30:21.263 INFO:teuthology.orchestra.run.smithi107.stdout:5452 punch from 0x845be00 to 0x846b800, (0xfa00 bytes) 2024-09-11T09:30:21.270 INFO:teuthology.orchestra.run.smithi107.stdout:5453 trunc from 0xbc8be00 to 0x672ca00 2024-09-11T09:30:21.286 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /var/log/ceph/exp.3.3.0 2024-09-11T09:30:21.298 INFO:teuthology.orchestra.run.smithi107.stdout:5454 read 0x5be0a00 thru 0x5bee2c6 (0xd8c7 bytes) 2024-09-11T09:30:21.305 INFO:teuthology.orchestra.run.smithi107.stdout:5457 punch from 0x82ac00 to 0x834600, (0x9a00 bytes) 2024-09-11T09:30:21.305 INFO:teuthology.orchestra.run.smithi107.stdout:5458 read 0x42ba600 thru 0x42c935d (0xed5e bytes) 2024-09-11T09:30:21.307 INFO:teuthology.orchestra.run.smithi107.stdout:5460 punch from 0x4961c00 to 0x4962a00, (0xe00 bytes) 2024-09-11T09:30:21.307 INFO:teuthology.orchestra.run.smithi107.stdout:5462 write 0x2444c00 thru 0x2448b2a (0x3f2b bytes) 2024-09-11T09:30:21.308 INFO:teuthology.orchestra.run.smithi107.stdout:5467 read 0x1c48400 thru 0x1c57d35 (0xf936 bytes) 2024-09-11T09:30:21.318 INFO:teuthology.orchestra.run.smithi107.stdout:5470 trunc from 0x672ca00 to 0xe3a4e00 2024-09-11T09:30:21.328 INFO:teuthology.orchestra.run.smithi107.stdout:5472 punch from 0x2f0a00 to 0x2f2e00, (0x2400 bytes) 2024-09-11T09:30:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:5474 read 0x524c400 thru 0x5250ec3 (0x4ac4 bytes) 2024-09-11T09:30:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:5476 write 0x5363800 thru 0x5370977 (0xd178 bytes) 2024-09-11T09:30:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:5477 read 0x9839e00 thru 0x9841786 (0x7987 bytes) 2024-09-11T09:30:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:5478 write 0xe370600 thru 0xe373b3e (0x353f bytes) 2024-09-11T09:30:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:5479 read 0x4dc4e00 thru 0x4dd09ba (0xbbbb bytes) 2024-09-11T09:30:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:5480 write 0x9388600 thru 0x938a306 (0x1d07 bytes) 2024-09-11T09:30:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:5481 write 0x90df600 thru 0x90df91d (0x31e bytes) 2024-09-11T09:30:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:5483 write 0x76a5800 thru 0x76ae13b (0x893c bytes) 2024-09-11T09:30:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:5485 read 0xa131600 thru 0xa1353bc (0x3dbd bytes) 2024-09-11T09:30:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:5486 punch from 0x8323800 to 0x8332c00, (0xf400 bytes) 2024-09-11T09:30:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:5487 punch from 0x2fe6600 to 0x2fea000, (0x3a00 bytes) 2024-09-11T09:30:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:5489 punch from 0xc33e800 to 0xc340800, (0x2000 bytes) 2024-09-11T09:30:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:5490 read 0xc9d4c00 thru 0xc9d55c5 (0x9c6 bytes) 2024-09-11T09:30:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:5491 read 0xdd51c00 thru 0xdd5c026 (0xa427 bytes) 2024-09-11T09:30:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:5492 read 0xb17b800 thru 0xb18a10e (0xe90f bytes) 2024-09-11T09:30:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:5493 punch from 0x4161400 to 0x416be00, (0xaa00 bytes) 2024-09-11T09:30:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:5495 read 0xaa89a00 thru 0xaa98f73 (0xf574 bytes) 2024-09-11T09:30:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:5496 read 0x2e96e00 thru 0x2e97fe8 (0x11e9 bytes) 2024-09-11T09:30:21.349 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:21.349 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:21.357 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:21.357+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:21.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:5499 write 0xebba400 thru 0xebc2c53 (0x8854 bytes) 2024-09-11T09:30:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:5501 trunc from 0xebc2e00 to 0xaec9400 2024-09-11T09:30:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:5504 read 0x7e82c00 thru 0x7e82ed5 (0x2d6 bytes) 2024-09-11T09:30:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:5505 read 0x868f200 thru 0x869d005 (0xde06 bytes) 2024-09-11T09:30:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:5507 clone 39 order 18 su 131072 sc 5 2024-09-11T09:30:21.459 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:21.459+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:21.468 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:21.468 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:21.477 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:21.560 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:21.561+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:21.619 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:21.619 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:21.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:21.669 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:21.669+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:21.762 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:21.763+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:21.771 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:21.771 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:21.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:21.863 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:21.864+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:21.881 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:21.881 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:21.890 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:21.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:30:21.964 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:21.965+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone38 from 0xaec9400 (overlap 0x91e00) to 0x48e00 2024-09-11T09:30:22.007 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.007 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.017 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:22.065 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:22.065+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:22.129 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.129 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.138 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:22.166 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:22.166+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:22.267 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:22.268+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:22.287 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.287 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:22.297 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-11T09:30:22.368 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:22.368+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:22.433 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.433 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:22.468 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:22.469+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:22.560 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.561 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.569 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:22.570 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:22.569+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:22.633 INFO:teuthology.orchestra.run.smithi107.stdout:5511 trunc from 0xaec9400 to 0x6c3e00 2024-09-11T09:30:22.670 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:22.670+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:22.676 INFO:teuthology.orchestra.run.smithi107.stdout:5512 write 0x1279600 thru 0x1287b5a (0xe55b bytes) 2024-09-11T09:30:22.677 INFO:teuthology.orchestra.run.smithi107.stdout:5514 punch from 0xf9f600 to 0xfadc00, (0xe600 bytes) 2024-09-11T09:30:22.678 INFO:teuthology.orchestra.run.smithi107.stdout:5516 read 0x1241600 thru 0x1247942 (0x6343 bytes) 2024-09-11T09:30:22.678 INFO:teuthology.orchestra.run.smithi107.stdout:5517 read 0xb26a00 thru 0xb2fc22 (0x9223 bytes) 2024-09-11T09:30:22.678 INFO:teuthology.orchestra.run.smithi107.stdout:5524 read 0xf79600 thru 0xf7e64f (0x5050 bytes) 2024-09-11T09:30:22.678 INFO:teuthology.orchestra.run.smithi107.stdout:5526 punch from 0xaf2600 to 0xaf4800, (0x2200 bytes) 2024-09-11T09:30:22.691 INFO:teuthology.orchestra.run.smithi107.stdout:5527 trunc from 0x1287c00 to 0xbdd8000 2024-09-11T09:30:22.700 INFO:teuthology.orchestra.run.smithi107.stdout:5530 write 0x3386800 thru 0x338b3e7 (0x4be8 bytes) 2024-09-11T09:30:22.701 INFO:teuthology.orchestra.run.smithi107.stdout:5532 write 0x2735600 thru 0x2739cbe (0x46bf bytes) 2024-09-11T09:30:22.701 INFO:teuthology.orchestra.run.smithi107.stdout:5533 punch from 0x51d7400 to 0x51da400, (0x3000 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5534 write 0xaba6000 thru 0xaba9f48 (0x3f49 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5535 read 0xb025000 thru 0xb028aa6 (0x3aa7 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5537 write 0xab81a00 thru 0xab89214 (0x7815 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5538 write 0xa504a00 thru 0xa50b813 (0x6e14 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5541 read 0x9c76800 thru 0x9c7c94f (0x6150 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5542 read 0x2684a00 thru 0x268b921 (0x6f22 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5544 read 0x784e000 thru 0x7854ca8 (0x6ca9 bytes) 2024-09-11T09:30:22.703 INFO:teuthology.orchestra.run.smithi107.stdout:5546 punch from 0x4916c00 to 0x491fa00, (0x8e00 bytes) 2024-09-11T09:30:22.704 INFO:teuthology.orchestra.run.smithi107.stdout:5547 read 0x2d11200 thru 0x2d1f864 (0xe665 bytes) 2024-09-11T09:30:22.704 INFO:teuthology.orchestra.run.smithi107.stdout:5549 clone 40 order 22 su 2097152 sc 6 2024-09-11T09:30:22.719 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.719 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.728 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:22.771 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:22.771+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:22.853 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:22.871 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:22.872+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:22.875 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:22.875 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:22.884 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:22.901 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:30:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:22.972 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:22.973+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:22.973 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone39 from 0xbdd8000 (overlap 0x6c3e00) to 0x0 2024-09-11T09:30:22.983 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:22.983 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:23.005 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:23.005 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:23.013 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:23.013 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:23.024 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:23.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:23.042 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:23.043 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:23.059 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:23.059 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:23.062 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:23.063 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:23.111 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:23.112+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:23.172 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:23.173 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:23.173 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:23.174 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:23.175+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:23.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:23.275 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:23.275+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:23.329 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:23.329 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:23.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:23.329 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:23.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.329 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 112.89125050600001, 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:23.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:23.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:23.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:23.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:23.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:23.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:23.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.275344779000001, 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:23.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:23.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:23.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:23.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:23.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:23.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:23.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:23.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:23.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:23.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.275147078000003, 2024-09-11T09:30:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:23.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:23.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:23.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:23.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:23.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:23.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:23.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:23.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:23.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.359 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.274888652000001, 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:23.360 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:23.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.362 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:23.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:23.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:23.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:23.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:23.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:23.368 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.274858768000001, 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.369 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:23.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.274325615999999, 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:23.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:23.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:23.379 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.273448166999998, 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:23.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.384 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.273409201, 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:23.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.273365724999998, 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.390 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.394 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.273241108000001, 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:23.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:23.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:23.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:23.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.272675638000003, 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.402 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:23.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:23.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.272645001999997, 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:23.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:23.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.411 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.270508569999997, 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.268988082, 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:23.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:23.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.420 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.264914253000001, 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:23.425 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.264800459, 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.426 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:23.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.264715776999999, 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:23.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.264669785999999, 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.439 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:23.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:23.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.264298488000001, 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 36.264156538999998, 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.453 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:23.454 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-11T09:30:23.454 INFO:teuthology.orchestra.run.smithi107.stdout:5550 write 0xc1f7400 thru 0xc1f7ec1 (0xac2 bytes) 2024-09-11T09:30:23.454 INFO:teuthology.orchestra.run.smithi107.stdout:5551 read 0x83d8e00 thru 0x83e0064 (0x7265 bytes) 2024-09-11T09:30:23.455 INFO:teuthology.orchestra.run.smithi107.stdout:5554 trunc from 0xc1f8000 to 0x6d62400 2024-09-11T09:30:23.455 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:23.456 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:23.455+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:23.473 INFO:teuthology.orchestra.run.smithi107.stdout:5555 trunc from 0x6d62400 to 0xd47c200 2024-09-11T09:30:23.484 INFO:teuthology.orchestra.run.smithi107.stdout:5556 write 0x3e9c800 thru 0x3ea520a (0x8a0b bytes) 2024-09-11T09:30:23.498 INFO:teuthology.orchestra.run.smithi107.stdout:5557 trunc from 0xd47c200 to 0x24dd200 2024-09-11T09:30:23.528 INFO:teuthology.orchestra.run.smithi107.stdout:5558 trunc from 0x24dd200 to 0x3e4b400 2024-09-11T09:30:23.536 INFO:teuthology.orchestra.run.smithi107.stdout:5563 punch from 0x202c000 to 0x203ae00, (0xee00 bytes) 2024-09-11T09:30:23.555 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:23.555+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:23.558 INFO:teuthology.orchestra.run.smithi107.stdout:5567 write 0x8926400 thru 0x893137e (0xaf7f bytes) 2024-09-11T09:30:23.570 INFO:teuthology.orchestra.run.smithi107.stdout:5568 trunc from 0x8931400 to 0xd7e6c00 2024-09-11T09:30:23.581 INFO:teuthology.orchestra.run.smithi107.stdout:5570 read 0xa1fe200 thru 0xa1ff75e (0x155f bytes) 2024-09-11T09:30:23.582 INFO:teuthology.orchestra.run.smithi107.stdout:5571 read 0xd6b2400 thru 0xd6b60dc (0x3cdd bytes) 2024-09-11T09:30:23.582 INFO:teuthology.orchestra.run.smithi107.stdout:5572 write 0x919f400 thru 0x91a8e13 (0x9a14 bytes) 2024-09-11T09:30:23.582 INFO:teuthology.orchestra.run.smithi107.stdout:5574 punch from 0x801e600 to 0x8028a00, (0xa400 bytes) 2024-09-11T09:30:23.583 INFO:teuthology.orchestra.run.smithi107.stdout:5575 read 0x4bf5e00 thru 0x4c0177a (0xb97b bytes) 2024-09-11T09:30:23.583 INFO:teuthology.orchestra.run.smithi107.stdout:5577 write 0x1249200 thru 0x124a917 (0x1718 bytes) 2024-09-11T09:30:23.583 INFO:teuthology.orchestra.run.smithi107.stdout:5578 write 0xadb9400 thru 0xadbd8cf (0x44d0 bytes) 2024-09-11T09:30:23.591 INFO:teuthology.orchestra.run.smithi107.stdout:5580 trunc from 0xd7e6c00 to 0x6b6ae00 2024-09-11T09:30:23.609 INFO:teuthology.orchestra.run.smithi107.stdout:5584 read 0x14fb800 thru 0x1506aa3 (0xb2a4 bytes) 2024-09-11T09:30:23.610 INFO:teuthology.orchestra.run.smithi107.stdout:5585 read 0x32b0e00 thru 0x32b8ece (0x80cf bytes) 2024-09-11T09:30:23.610 INFO:teuthology.orchestra.run.smithi107.stdout:5587 punch from 0x65f7a00 to 0x6600800, (0x8e00 bytes) 2024-09-11T09:30:23.611 INFO:teuthology.orchestra.run.smithi107.stdout:5588 read 0x5a47600 thru 0x5a51f6a (0xa96b bytes) 2024-09-11T09:30:23.611 INFO:teuthology.orchestra.run.smithi107.stdout:5589 read 0x185da00 thru 0x1861068 (0x3669 bytes) 2024-09-11T09:30:23.611 INFO:teuthology.orchestra.run.smithi107.stdout:5590 read 0x3bd0200 thru 0x3bd4bdd (0x49de bytes) 2024-09-11T09:30:23.616 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:23.616 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:23.617 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:23.617 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:23.625 INFO:teuthology.orchestra.run.smithi107.stdout:5591 write 0x8e9b000 thru 0x8e9c10a (0x110b bytes) 2024-09-11T09:30:23.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:23.631 INFO:teuthology.orchestra.run.smithi107.stdout:5592 punch from 0x5e32200 to 0x5e36e00, (0x4c00 bytes) 2024-09-11T09:30:23.631 INFO:teuthology.orchestra.run.smithi107.stdout:5594 punch from 0x7903600 to 0x7911000, (0xda00 bytes) 2024-09-11T09:30:23.646 INFO:teuthology.orchestra.run.smithi107.stdout:5595 write 0xe01bc00 thru 0xe0280f0 (0xc4f1 bytes) 2024-09-11T09:30:23.647 INFO:teuthology.orchestra.run.smithi107.stdout:5598 write 0x8ee2a00 thru 0x8eec8ea (0x9eeb bytes) 2024-09-11T09:30:23.648 INFO:teuthology.orchestra.run.smithi107.stdout:5600 read 0x713a600 thru 0x7144725 (0xa126 bytes) 2024-09-11T09:30:23.648 INFO:teuthology.orchestra.run.smithi107.stdout:5603 read 0x3a21800 thru 0x3a22f56 (0x1757 bytes) 2024-09-11T09:30:23.655 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:23.656+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:23.659 INFO:teuthology.orchestra.run.smithi107.stdout:5607 trunc from 0xe028200 to 0x4f63400 2024-09-11T09:30:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:5608 read 0x3fae400 thru 0x3fb656e (0x816f bytes) 2024-09-11T09:30:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:5609 punch from 0x1da3400 to 0x1daf800, (0xc400 bytes) 2024-09-11T09:30:23.698 INFO:teuthology.orchestra.run.smithi107.stdout:5611 write 0xc99ba00 thru 0xc9ab041 (0xf642 bytes) 2024-09-11T09:30:23.699 INFO:teuthology.orchestra.run.smithi107.stdout:5615 punch from 0x81d1c00 to 0x81ddc00, (0xc000 bytes) 2024-09-11T09:30:23.711 INFO:teuthology.orchestra.run.smithi107.stdout:5616 trunc from 0xc9ab200 to 0x34b4600 2024-09-11T09:30:23.747 INFO:teuthology.orchestra.run.smithi107.stdout:5617 write 0xc02ba00 thru 0xc039a7e (0xe07f bytes) 2024-09-11T09:30:23.748 INFO:teuthology.orchestra.run.smithi107.stdout:5618 punch from 0x755d800 to 0x7562400, (0x4c00 bytes) 2024-09-11T09:30:23.756 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:23.757+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:23.760 INFO:teuthology.orchestra.run.smithi107.stdout:5622 trunc from 0xc039c00 to 0x20d7600 2024-09-11T09:30:23.786 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:23.786 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:23.787 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:23.787 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:23.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:23.810 INFO:teuthology.orchestra.run.smithi107.stdout:5623 write 0xd5ac200 thru 0xd5b44ce (0x82cf bytes) 2024-09-11T09:30:23.811 INFO:teuthology.orchestra.run.smithi107.stdout:5624 punch from 0x246d400 to 0x2477200, (0x9e00 bytes) 2024-09-11T09:30:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:5628 read 0xd28c600 thru 0xd2906d0 (0x40d1 bytes) 2024-09-11T09:30:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:5630 write 0x5f94e00 thru 0x5f9dd32 (0x8f33 bytes) 2024-09-11T09:30:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:5631 write 0xc9e3200 thru 0xc9e4ec2 (0x1cc3 bytes) 2024-09-11T09:30:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:5632 punch from 0x8a8ac00 to 0x8a8c000, (0x1400 bytes) 2024-09-11T09:30:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:5635 read 0x41bd000 thru 0x41bf830 (0x2831 bytes) 2024-09-11T09:30:23.813 INFO:teuthology.orchestra.run.smithi107.stdout:5637 write 0x8b7fc00 thru 0x8b810dd (0x14de bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5641 punch from 0x5f81400 to 0x5f8e200, (0xce00 bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5642 punch from 0x6f8e00 to 0x6fe400, (0x5600 bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5643 write 0xb068400 thru 0xb06a97c (0x257d bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5644 write 0x4495c00 thru 0x44a463d (0xea3e bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5648 write 0x9843600 thru 0x984eb31 (0xb532 bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5649 write 0x6f2ae00 thru 0x6f31731 (0x6932 bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5652 write 0xcbc400 thru 0xcbcf71 (0xb72 bytes) 2024-09-11T09:30:23.814 INFO:teuthology.orchestra.run.smithi107.stdout:5653 punch from 0x5c1da00 to 0x5c29800, (0xbe00 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5656 read 0x40b6e00 thru 0x40b8536 (0x1737 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5657 read 0x58c3600 thru 0x58c7464 (0x3e65 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5659 write 0x5b28c00 thru 0x5b31a50 (0x8e51 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5660 read 0xae0cc00 thru 0xae0fee8 (0x32e9 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5661 punch from 0x829c00 to 0x835000, (0xb400 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5663 read 0x71ec000 thru 0x71f6500 (0xa501 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5667 punch from 0x4f82e00 to 0x4f90a00, (0xdc00 bytes) 2024-09-11T09:30:23.816 INFO:teuthology.orchestra.run.smithi107.stdout:5668 read 0x2b7d600 thru 0x2b8685a (0x925b bytes) 2024-09-11T09:30:23.817 INFO:teuthology.orchestra.run.smithi107.stdout:5669 write 0x4924000 thru 0x492e81d (0xa81e bytes) 2024-09-11T09:30:23.817 INFO:teuthology.orchestra.run.smithi107.stdout:5670 write 0xa5bd000 thru 0xa5c2cf2 (0x5cf3 bytes) 2024-09-11T09:30:23.817 INFO:teuthology.orchestra.run.smithi107.stdout:5671 read 0x696a400 thru 0x69770a7 (0xcca8 bytes) 2024-09-11T09:30:23.820 INFO:teuthology.orchestra.run.smithi107.stdout:5672 trunc from 0xd5b4600 to 0x9ff8200 2024-09-11T09:30:23.857 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:23.858+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:23.861 INFO:teuthology.orchestra.run.smithi107.stdout:5673 trunc from 0x9ff8200 to 0x5bf9200 2024-09-11T09:30:23.891 INFO:teuthology.orchestra.run.smithi107.stdout:5674 trunc from 0x5bf9200 to 0xbf23400 2024-09-11T09:30:23.911 INFO:teuthology.orchestra.run.smithi107.stdout:5675 trunc from 0xbf23400 to 0x26a2a00 2024-09-11T09:30:23.912 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572855734999999, 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572832516, 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572805355, 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.57278483, 2024-09-11T09:30:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572758396999999, 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572730245000001, 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572699865000001, 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572676790999999, 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.57264743, 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572626204000001, 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572605911, 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572579864, 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572558282999999, 2024-09-11T09:30:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572537479999999, 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572518069999999, 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572481102999999, 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572448701000001, 2024-09-11T09:30:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572426534, 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572401554000001, 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 14.572381175, 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.014 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:24.015 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:24.015+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:24.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:24.017 INFO:teuthology.orchestra.run.smithi107.stdout:5682 punch from 0x127ce00 to 0x1284a00, (0x7c00 bytes) 2024-09-11T09:30:24.017 INFO:teuthology.orchestra.run.smithi107.stdout:5683 punch from 0x1385200 to 0x1393600, (0xe400 bytes) 2024-09-11T09:30:24.017 INFO:teuthology.orchestra.run.smithi107.stdout:5685 trunc from 0x26a2a00 to 0x4db2200 2024-09-11T09:30:24.017 INFO:teuthology.orchestra.run.smithi107.stdout:5686 read 0x2d97a00 thru 0x2d9db6c (0x616d bytes) 2024-09-11T09:30:24.017 INFO:teuthology.orchestra.run.smithi107.stdout:5687 write 0xbf64400 thru 0xbf669b5 (0x25b6 bytes) 2024-09-11T09:30:24.017 INFO:teuthology.orchestra.run.smithi107.stdout:5688 write 0x98f9200 thru 0x9902d9a (0x9b9b bytes) 2024-09-11T09:30:24.018 INFO:teuthology.orchestra.run.smithi107.stdout:5690 trunc from 0xbf66a00 to 0x91bb000 2024-09-11T09:30:24.018 INFO:teuthology.orchestra.run.smithi107.stdout:5691 write 0x7d2c000 thru 0x7d31923 (0x5924 bytes) 2024-09-11T09:30:24.018 INFO:teuthology.orchestra.run.smithi107.stdout:5692 punch from 0x7df3800 to 0x7dffe00, (0xc600 bytes) 2024-09-11T09:30:24.018 INFO:teuthology.orchestra.run.smithi107.stdout:5693 trunc from 0x91bb000 to 0x2479200 2024-09-11T09:30:24.018 INFO:teuthology.orchestra.run.smithi107.stdout:5694 punch from 0x4c8200 to 0x4cd200, (0x5000 bytes) 2024-09-11T09:30:24.018 INFO:teuthology.orchestra.run.smithi107.stdout:5695 read 0x407c00 thru 0x4137b4 (0xbbb5 bytes) 2024-09-11T09:30:24.021 INFO:teuthology.orchestra.run.smithi107.stdout:5697 punch from 0x225c000 to 0x226aa00, (0xea00 bytes) 2024-09-11T09:30:24.021 INFO:teuthology.orchestra.run.smithi107.stdout:5699 read 0x17d7000 thru 0x17d92b5 (0x22b6 bytes) 2024-09-11T09:30:24.022 INFO:teuthology.orchestra.run.smithi107.stdout:5701 punch from 0x1fb6800 to 0x1fbc000, (0x5800 bytes) 2024-09-11T09:30:24.022 INFO:teuthology.orchestra.run.smithi107.stdout:5704 trunc from 0x2479200 to 0x26a5600 2024-09-11T09:30:24.051 INFO:teuthology.orchestra.run.smithi107.stdout:5709 write 0x4cf1600 thru 0x4cff1c6 (0xdbc7 bytes) 2024-09-11T09:30:24.057 INFO:teuthology.orchestra.run.smithi107.stdout:5710 punch from 0x24fb400 to 0x2509a00, (0xe600 bytes) 2024-09-11T09:30:24.058 INFO:teuthology.orchestra.run.smithi107.stdout:5711 read 0x3e3be00 thru 0x3e41320 (0x5521 bytes) 2024-09-11T09:30:24.058 INFO:teuthology.orchestra.run.smithi107.stdout:5712 punch from 0x2bfcc00 to 0x2c04800, (0x7c00 bytes) 2024-09-11T09:30:24.078 INFO:teuthology.orchestra.run.smithi107.stdout:5714 write 0xd710c00 thru 0xd71ee7b (0xe27c bytes) 2024-09-11T09:30:24.078 INFO:teuthology.orchestra.run.smithi107.stdout:5715 punch from 0x71d5a00 to 0x71df200, (0x9800 bytes) 2024-09-11T09:30:24.079 INFO:teuthology.orchestra.run.smithi107.stdout:5718 write 0x1443800 thru 0x144e52c (0xad2d bytes) 2024-09-11T09:30:24.080 INFO:teuthology.orchestra.run.smithi107.stdout:5719 read 0x6bf3e00 thru 0x6c02b55 (0xed56 bytes) 2024-09-11T09:30:24.080 INFO:teuthology.orchestra.run.smithi107.stdout:5720 write 0x2b2de00 thru 0x2b3ac75 (0xce76 bytes) 2024-09-11T09:30:24.085 INFO:teuthology.orchestra.run.smithi107.stdout:5721 trunc from 0xd71f000 to 0x9c5d200 2024-09-11T09:30:24.105 INFO:teuthology.orchestra.run.smithi107.stdout:5722 trunc from 0x9c5d200 to 0x7d88200 2024-09-11T09:30:24.115 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:24.116+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:24.115 INFO:teuthology.orchestra.run.smithi107.stdout:5723 write 0x683aa00 thru 0x683f6ee (0x4cef bytes) 2024-09-11T09:30:24.116 INFO:teuthology.orchestra.run.smithi107.stdout:5724 punch from 0x6868200 to 0x6875a00, (0xd800 bytes) 2024-09-11T09:30:24.117 INFO:teuthology.orchestra.run.smithi107.stdout:5725 read 0xd9b600 thru 0xda1ed4 (0x68d5 bytes) 2024-09-11T09:30:24.117 INFO:teuthology.orchestra.run.smithi107.stdout:5726 punch from 0x6b88c00 to 0x6b8e200, (0x5600 bytes) 2024-09-11T09:30:24.118 INFO:teuthology.orchestra.run.smithi107.stdout:5727 clone 41 order 25 su 8388608 sc 11 2024-09-11T09:30:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:24.133 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:24.142 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:24.216 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:24.217+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:24.284 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:24.285 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:24.285 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:24.294 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:24.317 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:24.317+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.414444231999994, 2024-09-11T09:30:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.414398188000007, 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.414386518000001, 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.409 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:24.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:24.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.413 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.413098249000001, 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.414 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:24.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:24.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:24.418 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.413062547999999, 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:24.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:24.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:24.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:24.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.413040511000005, 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:24.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.431 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.413004925999999, 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.432 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.433 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:24.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.436 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412944582999998, 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.437 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:24.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:24.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.441 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412895899000006, 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.442 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:24.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.446 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412839434999995, 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.447 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:24.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412819880000001, 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.452 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:24.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:24.456 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412772117000003, 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.457 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412658379000007, 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:24.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:24.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.466 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412639635000005, 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412600646000001, 2024-09-11T09:30:24.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:24.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:24.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.412577538999997, 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:24.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.704501047999997, 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:24.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:24.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:24.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.342101718000002, 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:24.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:24.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:24.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:24.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:24.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.336850640999998, 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:24.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:24.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:24.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.336590270999999, 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:24.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:24.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:24.504 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:24.506 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:24.506+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:24.506 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:24.607 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:24.608+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:24.618 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:24.618 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:24.618 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:24.618 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:24.627 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:24.708 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:24.709+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:24.733 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:24.733 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:24.733 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:24.733 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:24.733 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:24.742 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:24.809 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:24.810+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.939662158999994, 2024-09-11T09:30:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.939431587000001, 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.861 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.939334383000002, 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.862 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:24.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:24.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:24.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:24.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:24.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.939099619000004, 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.868 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:24.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:24.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.938903603, 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.873 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.938635165999997, 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.878 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:24.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:24.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:24.882 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.938533415999999, 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:24.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:24.886 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:24.887 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.938139758999995, 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.888 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:24.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.938013514000005, 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:24.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.937804131999997, 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.897 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:24.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.937693182999993, 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:24.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.904 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.937437244999998, 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.907 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:24.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:24.911 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.937284715999994, 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.912 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:24.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.937092351999993, 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.917 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:24.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:24.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:24.921 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.936761934000003, 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:24.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:24.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:24.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:24.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:24.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.928 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.928 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.928 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:24.929 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.936485603999998, 2024-09-11T09:30:24.929 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:24.929 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.929 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.929 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.930 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.930 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.930 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:24.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.931 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:24.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.932 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.932 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:24.932 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.933 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.934 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:24.934 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:24.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.934 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.934 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:24.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:24.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.935 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:24.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:24.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.936 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.936 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:24.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:24.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.937 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.937 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:24.937 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:24.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.938 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.938 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:24.938 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:24.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.939 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.939 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:24.939 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:24.939 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.939 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.940 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.940 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.940 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:24.940 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.936431683999999, 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.941 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:24.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.942 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.942 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.942 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:24.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.946 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.936231114999998, 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:24.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.951 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.935875576000001, 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:24.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 42.156610043000001, 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:24.957 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:24.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:24.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:24.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:24.964 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:24.965 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:30:24.966 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:24.966+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:24.966 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:24.999 INFO:teuthology.orchestra.run.smithi107.stdout:flattening image image_client.0-clone40 2024-09-11T09:30:25.064 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:25.066 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:25.066+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:25.119 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:25.119 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:25.119 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:25.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:25.167 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:25.167+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:25.179 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:25.196 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:25.197 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:25.215 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:25.215 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:25.234 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:25.235 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:25.243 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:25.244 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:25.251 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:25.252 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:25.253 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:25.270 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:25.270 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:25.270 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:25.270+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:25.274 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:25.274 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:25.347 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-11T09:30:25.362 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:25.362 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:25.369 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:25.370+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:25.371 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:25.403 INFO:teuthology.orchestra.run.smithi107.stdout:5730 write 0x5bd8600 thru 0x5bdf178 (0x6b79 bytes) 2024-09-11T09:30:25.404 INFO:teuthology.orchestra.run.smithi107.stdout:5732 punch from 0x4cdc400 to 0x4cdd600, (0x1200 bytes) 2024-09-11T09:30:25.431 INFO:teuthology.orchestra.run.smithi107.stdout:5733 punch from 0x3169c00 to 0x3173800, (0x9c00 bytes) 2024-09-11T09:30:25.434 INFO:teuthology.orchestra.run.smithi107.stdout:5734 read 0x6f08e00 thru 0x6f0bc57 (0x2e58 bytes) 2024-09-11T09:30:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:5736 write 0x9947e00 thru 0x99572ca (0xf4cb bytes) 2024-09-11T09:30:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:5737 read 0x5953200 thru 0x5956fa1 (0x3da2 bytes) 2024-09-11T09:30:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:5741 write 0x4801600 thru 0x4807765 (0x6166 bytes) 2024-09-11T09:30:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:5742 write 0x2982c00 thru 0x298583e (0x2c3f bytes) 2024-09-11T09:30:25.453 INFO:teuthology.orchestra.run.smithi107.stdout:5748 read 0x49fa600 thru 0x4a042fe (0x9cff bytes) 2024-09-11T09:30:25.458 INFO:teuthology.orchestra.run.smithi107.stdout:5750 trunc from 0x9957400 to 0x5626400 2024-09-11T09:30:25.470 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:25.470+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:25.481 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:25.481 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:25.491 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:25.500 INFO:teuthology.orchestra.run.smithi107.stdout:5752 trunc from 0x5626400 to 0xd3a4600 2024-09-11T09:30:25.509 INFO:teuthology.orchestra.run.smithi107.stdout:5753 read 0xaa37400 thru 0xaa3f37d (0x7f7e bytes) 2024-09-11T09:30:25.510 INFO:teuthology.orchestra.run.smithi107.stdout:5756 read 0x347be00 thru 0x347c1f1 (0x3f2 bytes) 2024-09-11T09:30:25.511 INFO:teuthology.orchestra.run.smithi107.stdout:5757 punch from 0x27fa800 to 0x2804e00, (0xa600 bytes) 2024-09-11T09:30:25.522 INFO:teuthology.orchestra.run.smithi107.stdout:5759 trunc from 0xd3a4600 to 0x48e6e00 2024-09-11T09:30:25.543 INFO:teuthology.orchestra.run.smithi107.stdout:5760 punch from 0xac2800 to 0xac9600, (0x6e00 bytes) 2024-09-11T09:30:25.546 INFO:teuthology.orchestra.run.smithi107.stdout:5762 read 0x809a00 thru 0x80fb71 (0x6172 bytes) 2024-09-11T09:30:25.548 INFO:teuthology.orchestra.run.smithi107.stdout:5765 write 0x17baa00 thru 0x17bef00 (0x4501 bytes) 2024-09-11T09:30:25.548 INFO:teuthology.orchestra.run.smithi107.stdout:5767 write 0x1c7fc00 thru 0x1c85a30 (0x5e31 bytes) 2024-09-11T09:30:25.551 INFO:teuthology.orchestra.run.smithi107.stdout:5769 trunc from 0x48e6e00 to 0x6f1e600 2024-09-11T09:30:25.571 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:25.571+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:25.578 INFO:teuthology.orchestra.run.smithi107.stdout:5770 write 0xe038600 thru 0xe03af0c (0x290d bytes) 2024-09-11T09:30:25.591 INFO:teuthology.orchestra.run.smithi107.stdout:5771 write 0xe80d600 thru 0xe8128f2 (0x52f3 bytes) 2024-09-11T09:30:25.593 INFO:teuthology.orchestra.run.smithi107.stdout:5772 punch from 0x401f000 to 0x4029600, (0xa600 bytes) 2024-09-11T09:30:25.594 INFO:teuthology.orchestra.run.smithi107.stdout:5775 write 0xd50f600 thru 0xd512d66 (0x3767 bytes) 2024-09-11T09:30:25.595 INFO:teuthology.orchestra.run.smithi107.stdout:5776 write 0x5aab600 thru 0x5ab9588 (0xdf89 bytes) 2024-09-11T09:30:25.595 INFO:teuthology.orchestra.run.smithi107.stdout:5777 clone 42 order 22 su 262144 sc 6 2024-09-11T09:30:25.641 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:25.641 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:25.649 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:25.693 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:25.694+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:25.752 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:25.752 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:25.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:25.802 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:25.802+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:25.874 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:25.875+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:25.904 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:25.904 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:25.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:25.954 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:30:25.975 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:25.976+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:26.021 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.021 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.027 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone41 from 0xe812a00 (overlap 0x48e6e00) to 0xe94800 2024-09-11T09:30:26.030 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:26.076 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:26.076+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:26.177 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:26.177+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:26.177 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.177 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:26.278 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:26.279+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:26.334 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.334 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:26.355 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-11T09:30:26.358 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:30:26.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:26.358 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-11T09:30:26.358 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T09:30:26.402 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:26.403+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:26.480 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:26.481+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:26.497 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.497 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.508 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:26.581 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:26.581+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:26.653 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.653 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:26.664 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:26.664 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 99, 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "23a8bf95-caba-4fea-a762-61040a1c1b77", 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-11T09:28:17.894525+0000", 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-11T09:30:25.952541+0000", 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2024-09-11T09:30:12.854131+0000", 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2024-09-11T09:29:50.757033+0000", 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5806080, 2024-09-11T09:30:26.665 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "nodeep-scrub", 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "noscrub", 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:26.666 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 5, 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 3, 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 6, 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "luminous", 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "jewel", 2024-09-11T09:30:26.667 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "squid", 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_crimson": false, 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": ".mgr", 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:28.765227+0000", 2024-09-11T09:30:26.668 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:26.669 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2024-09-11T09:30:26.670 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2024-09-11T09:30:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2024-09-11T09:30:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2024-09-11T09:30:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2024-09-11T09:30:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:26.671 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.672 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "15", 2024-09-11T09:30:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:26.673 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:26.674 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:26.675 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:26.676 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_max": 32, 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": {} 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 5.880000114440918, 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 5.880000114440918, 2024-09-11T09:30:26.677 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.17000000178813934, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:34.774979+0000", 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:26.678 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2024-09-11T09:30:26.679 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2024-09-11T09:30:26.680 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 2, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 19, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:26.681 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.682 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:26.683 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:26.683 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:26.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:26.684 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:26.684 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:26.684 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:26.685 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:26.685 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:26.685 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:26.685 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:26.685 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:26.685 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 2.2699999809265137, 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 2.2699999809265137, 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-11T09:30:26.686 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "pool_client.0", 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:45.573818+0000", 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:26.687 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 32, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 32, 2024-09-11T09:30:26.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 32, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 32, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 32, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "99", 2024-09-11T09:30:26.689 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 44, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 99, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:26.690 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.691 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:26.692 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 1.7000000476837158, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 1.7000000476837158, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:26.693 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "44ce15f5-731e-4d58-9dd7-bd8f186e5783", 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 0, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:26.694 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 11, 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 90, 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2024-09-11T09:30:26.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.697 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2024-09-11T09:30:26.697 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.698 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.698 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:26.698 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.698 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.699 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2024-09-11T09:30:26.699 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.699 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.699 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.701 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/4208334809", 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/4208334809", 2024-09-11T09:30:26.702 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/4208334809", 2024-09-11T09:30:26.703 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/4208334809", 2024-09-11T09:30:26.703 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:26.703 INFO:teuthology.orchestra.run.smithi017.stdout: "exists" 2024-09-11T09:30:26.703 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "dba82e38-a74b-4d42-bad4-9b1e013d4da3", 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 0.4687957763671875, 2024-09-11T09:30:26.704 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:26.705 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:26.705 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:26.705 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:26.705 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:26.705 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:26.705 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:26.707 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:26.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.710 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2024-09-11T09:30:26.710 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.711 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.711 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2024-09-11T09:30:26.711 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.711 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.712 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:26.712 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.712 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.712 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:26.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6817/361445038", 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6819/361445038", 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6823/361445038", 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6821/361445038", 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:26.714 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "64a33918-9bb2-4d59-a66e-73da15d78cbd", 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:26.715 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:26.716 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:26.716 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:26.716 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:26.716 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:26.716 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:26.716 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:26.717 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:26.717 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:26.717 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:26.717 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.718 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2024-09-11T09:30:26.718 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.718 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.718 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2024-09-11T09:30:26.719 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.719 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.719 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:26.719 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.720 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.720 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2024-09-11T09:30:26.720 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.721 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2024-09-11T09:30:26.721 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.721 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.722 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/211521567", 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/211521567", 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/211521567", 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/211521567", 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:26.724 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a4774bc9-54ea-4272-b6b4-a6dd00c4c930", 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:26.725 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6800", 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.726 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6801", 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6802", 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.727 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6803", 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6806", 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6807", 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6804", 2024-09-11T09:30:26.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6805", 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6801/2070567113", 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6803/2070567113", 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6807/2070567113", 2024-09-11T09:30:26.730 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6805/2070567113", 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a3984c6d-7250-4e85-a032-1dc0315642d8", 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:26.731 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:26.732 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:26.732 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:26.732 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:26.732 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:26.732 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:26.733 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:26.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6816", 2024-09-11T09:30:26.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.734 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.734 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6817", 2024-09-11T09:30:26.734 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.735 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.735 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:26.735 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6818", 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6819", 2024-09-11T09:30:26.736 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.737 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.738 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6822", 2024-09-11T09:30:26.738 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.738 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6823", 2024-09-11T09:30:26.739 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.739 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.739 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:26.739 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.740 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6820", 2024-09-11T09:30:26.740 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6821", 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.741 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6817/1795828331", 2024-09-11T09:30:26.742 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6819/1795828331", 2024-09-11T09:30:26.742 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6823/1795828331", 2024-09-11T09:30:26.742 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6821/1795828331", 2024-09-11T09:30:26.742 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:26.742 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:26.742 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "0a40bbb7-9d37-4a1b-b525-86608a3eaaa2", 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:26.743 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:26.744 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:26.744 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:26.744 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:26.744 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:26.744 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:26.744 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6808", 2024-09-11T09:30:26.745 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6809", 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.746 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6810", 2024-09-11T09:30:26.747 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6811", 2024-09-11T09:30:26.748 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.749 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:26.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6814", 2024-09-11T09:30:26.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6815", 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.751 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6812", 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6813", 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6809/3808455039", 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6811/3808455039", 2024-09-11T09:30:26.753 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6815/3808455039", 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6813/3808455039", 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.754 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:26.755 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2024-09-11T09:30:26.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.755 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:26.755 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:26.755 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:26.755 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:26.756 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:26.756 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:26.756 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.320025+0000", 2024-09-11T09:30:26.756 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 90 2024-09-11T09:30:26.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:26.757 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:26.757 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:26.757 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:26.757 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.922939+0000", 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:26.758 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.876780+0000", 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.854159+0000", 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.081538+0000", 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:26.760 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.168719+0000", 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [], 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_primaries": [], 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklist": {}, 2024-09-11T09:30:26.761 INFO:teuthology.orchestra.run.smithi017.stdout: "range_blocklist": {}, 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [], 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [], 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2024-09-11T09:30:26.762 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {}, 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": { 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_enabled": false, 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_bucket_count": 0, 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "degraded_stretch_mode": 0, 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "recovering_stretch_mode": 0, 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_bucket": 0 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:26.763 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:26.765 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:26.765+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:26.766 INFO:teuthology.orchestra.run.smithi107.stdout:5780 read 0x78d1400 thru 0x78ded0b (0xd90c bytes) 2024-09-11T09:30:26.766 INFO:teuthology.orchestra.run.smithi107.stdout:5781 write 0x3d57200 thru 0x3d63caf (0xcab0 bytes) 2024-09-11T09:30:26.766 INFO:teuthology.orchestra.run.smithi107.stdout:5782 punch from 0x3321200 to 0x3328e00, (0x7c00 bytes) 2024-09-11T09:30:26.766 INFO:teuthology.orchestra.run.smithi107.stdout:5784 write 0x153000 thru 0x1610cf (0xe0d0 bytes) 2024-09-11T09:30:26.766 INFO:teuthology.orchestra.run.smithi107.stdout:5788 trunc from 0xe812a00 to 0x4889800 2024-09-11T09:30:26.767 INFO:tasks.thrashosds.thrasher:j is {'epoch': 99, 'fsid': '23a8bf95-caba-4fea-a762-61040a1c1b77', 'created': '2024-09-11T09:28:17.894525+0000', 'modified': '2024-09-11T09:30:25.952541+0000', 'last_up_change': '2024-09-11T09:30:12.854131+0000', 'last_in_change': '2024-09-11T09:29:50.757033+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T09:28:28.765227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T09:28:34.774979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-11T09:28:45.573818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 44, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}], 'osds': [{'osd': 0, 'uuid': '44ce15f5-731e-4d58-9dd7-bd8f186e5783', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 90, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6800', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6801', 'nonce': 4208334809}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6802', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6803', 'nonce': 4208334809}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6806', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6807', 'nonce': 4208334809}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6804', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6805', 'nonce': 4208334809}]}, 'public_addr': '172.21.15.17:6801/4208334809', 'cluster_addr': '172.21.15.17:6803/4208334809', 'heartbeat_back_addr': '172.21.15.17:6807/4208334809', 'heartbeat_front_addr': '172.21.15.17:6805/4208334809', 'state': ['exists']}, {'osd': 1, 'uuid': 'dba82e38-a74b-4d42-bad4-9b1e013d4da3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4687957763671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6816', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6817', 'nonce': 361445038}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6818', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6819', 'nonce': 361445038}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6822', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6823', 'nonce': 361445038}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6820', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6821', 'nonce': 361445038}]}, 'public_addr': '172.21.15.17:6817/361445038', 'cluster_addr': '172.21.15.17:6819/361445038', 'heartbeat_back_addr': '172.21.15.17:6823/361445038', 'heartbeat_front_addr': '172.21.15.17:6821/361445038', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64a33918-9bb2-4d59-a66e-73da15d78cbd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6808', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6809', 'nonce': 211521567}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6810', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6811', 'nonce': 211521567}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6814', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6815', 'nonce': 211521567}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6812', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6813', 'nonce': 211521567}]}, 'public_addr': '172.21.15.17:6809/211521567', 'cluster_addr': '172.21.15.17:6811/211521567', 'heartbeat_back_addr': '172.21.15.17:6815/211521567', 'heartbeat_front_addr': '172.21.15.17:6813/211521567', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a4774bc9-54ea-4272-b6b4-a6dd00c4c930', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 2070567113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 2070567113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 2070567113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 2070567113}]}, 'public_addr': '172.21.15.96:6801/2070567113', 'cluster_addr': '172.21.15.96:6803/2070567113', 'heartbeat_back_addr': '172.21.15.96:6807/2070567113', 'heartbeat_front_addr': '172.21.15.96:6805/2070567113', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a3984c6d-7250-4e85-a032-1dc0315642d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 1795828331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 1795828331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 1795828331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 1795828331}]}, 'public_addr': '172.21.15.96:6817/1795828331', 'cluster_addr': '172.21.15.96:6819/1795828331', 'heartbeat_back_addr': '172.21.15.96:6823/1795828331', 'heartbeat_front_addr': '172.21.15.96:6821/1795828331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0a40bbb7-9d37-4a1b-b525-86608a3eaaa2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 3808455039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 3808455039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 3808455039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 3808455039}]}, 'public_addr': '172.21.15.96:6809/3808455039', 'cluster_addr': '172.21.15.96:6811/3808455039', 'heartbeat_back_addr': '172.21.15.96:6815/3808455039', 'heartbeat_front_addr': '172.21.15.96:6813/3808455039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.320025+0000', 'dead_epoch': 90}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.922939+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.876780+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.854159+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.081538+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.168719+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T09:30:26.767 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:30:26.774 INFO:teuthology.orchestra.run.smithi107.stdout:5793 trunc from 0x4889800 to 0xcb35000 2024-09-11T09:30:26.775 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.775 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.784 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:26.798 INFO:teuthology.orchestra.run.smithi107.stdout:5794 write 0xde92800 thru 0xde9bbfb (0x93fc bytes) 2024-09-11T09:30:26.799 INFO:teuthology.orchestra.run.smithi107.stdout:5801 write 0x3fac000 thru 0x3fb5946 (0x9947 bytes) 2024-09-11T09:30:26.799 INFO:teuthology.orchestra.run.smithi107.stdout:5803 read 0x81b6e00 thru 0x81c2d42 (0xbf43 bytes) 2024-09-11T09:30:26.800 INFO:teuthology.orchestra.run.smithi107.stdout:5804 read 0x8655a00 thru 0x8660dda (0xb3db bytes) 2024-09-11T09:30:26.800 INFO:teuthology.orchestra.run.smithi107.stdout:5805 write 0x5257200 thru 0x525e835 (0x7636 bytes) 2024-09-11T09:30:26.800 INFO:teuthology.orchestra.run.smithi107.stdout:5807 write 0x7a94200 thru 0x7a9e112 (0x9f13 bytes) 2024-09-11T09:30:26.800 INFO:teuthology.orchestra.run.smithi107.stdout:5810 write 0x3e5a600 thru 0x3e5ed61 (0x4762 bytes) 2024-09-11T09:30:26.800 INFO:teuthology.orchestra.run.smithi107.stdout:5813 write 0x48df600 thru 0x48ec774 (0xd175 bytes) 2024-09-11T09:30:26.801 INFO:teuthology.orchestra.run.smithi107.stdout:5814 read 0x5262e00 thru 0x526a32c (0x752d bytes) 2024-09-11T09:30:26.801 INFO:teuthology.orchestra.run.smithi107.stdout:5815 write 0x94e0c00 thru 0x94e86b4 (0x7ab5 bytes) 2024-09-11T09:30:26.801 INFO:teuthology.orchestra.run.smithi107.stdout:5817 punch from 0x3448000 to 0x344b800, (0x3800 bytes) 2024-09-11T09:30:26.801 INFO:teuthology.orchestra.run.smithi107.stdout:5821 read 0x4a85800 thru 0x4a95607 (0xfe08 bytes) 2024-09-11T09:30:26.802 INFO:teuthology.orchestra.run.smithi107.stdout:5822 read 0xc82200 thru 0xc8569e (0x349f bytes) 2024-09-11T09:30:26.802 INFO:teuthology.orchestra.run.smithi107.stdout:5824 read 0xb77c400 thru 0xb77d154 (0xd55 bytes) 2024-09-11T09:30:26.802 INFO:teuthology.orchestra.run.smithi107.stdout:5827 read 0xc76c00 thru 0xc83d2d (0xd12e bytes) 2024-09-11T09:30:26.804 INFO:teuthology.orchestra.run.smithi107.stdout:5828 punch from 0xc322000 to 0xc331800, (0xf800 bytes) 2024-09-11T09:30:26.804 INFO:teuthology.orchestra.run.smithi107.stdout:5829 read 0x9cd6800 thru 0x9cd7384 (0xb85 bytes) 2024-09-11T09:30:26.804 INFO:teuthology.orchestra.run.smithi107.stdout:5830 punch from 0x5cfe00 to 0x5d9200, (0x9400 bytes) 2024-09-11T09:30:26.804 INFO:teuthology.orchestra.run.smithi107.stdout:5832 punch from 0x7537800 to 0x753c600, (0x4e00 bytes) 2024-09-11T09:30:26.817 INFO:teuthology.orchestra.run.smithi107.stdout:5833 trunc from 0xde9bc00 to 0x263d400 2024-09-11T09:30:26.865 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:26.865+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:26.868 INFO:teuthology.orchestra.run.smithi107.stdout:5835 write 0x9baae00 thru 0x9bb9a69 (0xec6a bytes) 2024-09-11T09:30:26.870 INFO:teuthology.orchestra.run.smithi107.stdout:5836 trunc from 0x9bb9c00 to 0xb8a1200 2024-09-11T09:30:26.881 INFO:teuthology.orchestra.run.smithi107.stdout:5838 write 0x84de400 thru 0x84e34fc (0x50fd bytes) 2024-09-11T09:30:26.885 INFO:teuthology.orchestra.run.smithi107.stdout:5840 trunc from 0xb8a1200 to 0x9666000 2024-09-11T09:30:26.898 INFO:teuthology.orchestra.run.smithi107.stdout:5841 write 0x3b99200 thru 0x3b9976f (0x570 bytes) 2024-09-11T09:30:26.899 INFO:teuthology.orchestra.run.smithi107.stdout:5842 read 0x7d55400 thru 0x7d5c9bf (0x75c0 bytes) 2024-09-11T09:30:26.910 INFO:teuthology.orchestra.run.smithi107.stdout:5843 trunc from 0x9666000 to 0x458200 2024-09-11T09:30:26.942 INFO:teuthology.orchestra.run.smithi107.stdout:5847 read 0x59800 thru 0x668ca (0xd0cb bytes) 2024-09-11T09:30:26.963 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:26.964 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:26.965 INFO:teuthology.orchestra.run.smithi107.stdout:5848 write 0xb587000 thru 0xb590584 (0x9585 bytes) 2024-09-11T09:30:26.966 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:26.966+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:26.967 INFO:teuthology.orchestra.run.smithi107.stdout:5850 write 0x6ce4e00 thru 0x6ceee4a (0xa04b bytes) 2024-09-11T09:30:26.967 INFO:teuthology.orchestra.run.smithi107.stdout:5853 write 0x3e68a00 thru 0x3e6fb5a (0x715b bytes) 2024-09-11T09:30:26.967 INFO:teuthology.orchestra.run.smithi107.stdout:5854 read 0x4e2bc00 thru 0x4e35b91 (0x9f92 bytes) 2024-09-11T09:30:26.967 INFO:teuthology.orchestra.run.smithi107.stdout:5855 read 0xa18cc00 thru 0xa19401e (0x741f bytes) 2024-09-11T09:30:26.967 INFO:teuthology.orchestra.run.smithi107.stdout:5856 punch from 0xaf51400 to 0xaf5ce00, (0xba00 bytes) 2024-09-11T09:30:26.968 INFO:teuthology.orchestra.run.smithi107.stdout:5858 read 0x31aa000 thru 0x31b5834 (0xb835 bytes) 2024-09-11T09:30:26.968 INFO:teuthology.orchestra.run.smithi107.stdout:5859 write 0xb4b4a00 thru 0xb4be69c (0x9c9d bytes) 2024-09-11T09:30:26.971 INFO:teuthology.orchestra.run.smithi107.stdout:5860 trunc from 0xb590600 to 0xdcf9800 2024-09-11T09:30:26.973 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:26.982 INFO:teuthology.orchestra.run.smithi107.stdout:5861 punch from 0x5691400 to 0x5691c00, (0x800 bytes) 2024-09-11T09:30:26.984 INFO:teuthology.orchestra.run.smithi107.stdout:5865 punch from 0x4cdf200 to 0x4cec800, (0xd600 bytes) 2024-09-11T09:30:26.984 INFO:teuthology.orchestra.run.smithi107.stdout:5871 write 0x80c800 thru 0x81c476 (0xfc77 bytes) 2024-09-11T09:30:26.985 INFO:teuthology.orchestra.run.smithi107.stdout:5873 write 0x3088a00 thru 0x308f4bc (0x6abd bytes) 2024-09-11T09:30:26.985 INFO:teuthology.orchestra.run.smithi107.stdout:5874 write 0xc777200 thru 0xc77f2d6 (0x80d7 bytes) 2024-09-11T09:30:26.985 INFO:teuthology.orchestra.run.smithi107.stdout:5875 read 0x80a1c00 thru 0x80a38aa (0x1cab bytes) 2024-09-11T09:30:26.985 INFO:teuthology.orchestra.run.smithi107.stdout:5876 write 0x6c0e800 thru 0x6c132ad (0x4aae bytes) 2024-09-11T09:30:26.986 INFO:teuthology.orchestra.run.smithi107.stdout:5877 write 0x14cb800 thru 0x14d3f6c (0x876d bytes) 2024-09-11T09:30:26.986 INFO:teuthology.orchestra.run.smithi107.stdout:5878 trunc from 0xdcf9800 to 0xe9bea00 2024-09-11T09:30:26.999 INFO:teuthology.orchestra.run.smithi107.stdout:5880 read 0x9e57400 thru 0x9e6056d (0x916e bytes) 2024-09-11T09:30:27.000 INFO:teuthology.orchestra.run.smithi107.stdout:5883 write 0x3a6ba00 thru 0x3a78c50 (0xd251 bytes) 2024-09-11T09:30:27.000 INFO:teuthology.orchestra.run.smithi107.stdout:5884 write 0x231b800 thru 0x231c0ac (0x8ad bytes) 2024-09-11T09:30:27.009 INFO:teuthology.orchestra.run.smithi107.stdout:5887 trunc from 0xe9bea00 to 0x74da400 2024-09-11T09:30:27.028 INFO:teuthology.orchestra.run.smithi107.stdout:5889 read 0x2a0f600 thru 0x2a159ba (0x63bb bytes) 2024-09-11T09:30:27.029 INFO:teuthology.orchestra.run.smithi107.stdout:5890 trunc from 0x74da400 to 0x7714e00 2024-09-11T09:30:27.038 INFO:teuthology.orchestra.run.smithi107.stdout:5892 read 0x5756200 thru 0x57569ea (0x7eb bytes) 2024-09-11T09:30:27.039 INFO:teuthology.orchestra.run.smithi107.stdout:5893 punch from 0x47cf400 to 0x47d7e00, (0x8a00 bytes) 2024-09-11T09:30:27.040 INFO:teuthology.orchestra.run.smithi107.stdout:5894 read 0x253ba00 thru 0x25441ed (0x87ee bytes) 2024-09-11T09:30:27.042 INFO:teuthology.orchestra.run.smithi107.stdout:5896 trunc from 0x7714e00 to 0x8b21000 2024-09-11T09:30:27.050 INFO:teuthology.orchestra.run.smithi107.stdout:5898 read 0x3bf2400 thru 0x3bfdcd9 (0xb8da bytes) 2024-09-11T09:30:27.051 INFO:teuthology.orchestra.run.smithi107.stdout:5899 read 0x6c62200 thru 0x6c6e3bc (0xc1bd bytes) 2024-09-11T09:30:27.054 INFO:teuthology.orchestra.run.smithi107.stdout:5903 trunc from 0x8b21000 to 0xb2f8000 2024-09-11T09:30:27.067 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:27.067+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:27.075 INFO:teuthology.orchestra.run.smithi107.stdout:5905 write 0xd9eb600 thru 0xd9fb528 (0xff29 bytes) 2024-09-11T09:30:27.075 INFO:teuthology.orchestra.run.smithi107.stdout:5906 punch from 0x21c8000 to 0x21c8800, (0x800 bytes) 2024-09-11T09:30:27.080 INFO:teuthology.orchestra.run.smithi107.stdout:5907 trunc from 0xd9fb600 to 0xe08ca00 2024-09-11T09:30:27.090 INFO:teuthology.orchestra.run.smithi107.stdout:5908 read 0x2917a00 thru 0x291a174 (0x2775 bytes) 2024-09-11T09:30:27.092 INFO:teuthology.orchestra.run.smithi107.stdout:5909 write 0x2bb0800 thru 0x2bb381d (0x301e bytes) 2024-09-11T09:30:27.092 INFO:teuthology.orchestra.run.smithi107.stdout:5912 write 0x12dbc00 thru 0x12dbdb8 (0x1b9 bytes) 2024-09-11T09:30:27.093 INFO:teuthology.orchestra.run.smithi107.stdout:5913 write 0x2fa5e00 thru 0x2fa8aa2 (0x2ca3 bytes) 2024-09-11T09:30:27.093 INFO:teuthology.orchestra.run.smithi107.stdout:5914 read 0x9455000 thru 0x945c178 (0x7179 bytes) 2024-09-11T09:30:27.093 INFO:teuthology.orchestra.run.smithi107.stdout:5916 punch from 0x3214200 to 0x3218c00, (0x4a00 bytes) 2024-09-11T09:30:27.106 INFO:teuthology.orchestra.run.smithi107.stdout:5917 trunc from 0xe08ca00 to 0x3364a00 2024-09-11T09:30:27.126 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:27.127 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:30:27.129 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:27.129 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:27.139 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:27.141 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-11T09:30:26.740904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1288553932,"num_objects":2123,"num_object_clones":604,"num_object_copies":4246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2123,"num_whiteouts":827,"num_read":271519,"num_read_kb":1306687,"num_write":20836,"num_write_kb":417568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2823,"num_bytes_recovered":126375024,"num_keys_recovered":360,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17423,"ondisk_log_size":17423,"up":67,"acting":67,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":432760,"kb_used_data":61060,"kb_used_omap":9,"kb_used_meta":371574,"kb_avail":565798280,"statfs":{"total":579820584960,"available":579377438720,"internally_reserved":0,"allocated":62525440,"data_stored":463363365,"data_compressed":19618148,"data_compressed_allocated":37126144,"data_compressed_original":445109760,"omap_allocated":9568,"internal_metadata":380492448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":118795729,"num_objects":242,"num_object_clones":87,"num_object_copies":484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":82,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":242,"num_whiteouts":99,"num_read":17369,"num_read_kb":130742,"num_write":2491,"num_write_kb":33476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.844182"},"pg_stats":[{"pgid":"3.1f","version":"97'518","reported_seq":4245,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.425158+0000","last_change":"2024-09-11T09:30:13.866360+0000","last_active":"2024-09-11T09:30:23.425158+0000","last_peered":"2024-09-11T09:30:23.425158+0000","last_clean":"2024-09-11T09:30:12.119638+0000","last_became_active":"2024-09-11T09:30:13.866360+0000","last_became_peered":"2024-09-11T09:30:13.866360+0000","last_unstale":"2024-09-11T09:30:23.425158+0000","last_undegraded":"2024-09-11T09:30:13.864780+0000","last_fullsized":"2024-09-11T09:30:13.864710+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:16:43.153839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012367610000000001,"stat_sum":{"num_bytes":40512684,"num_objects":72,"num_object_clones":19,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":72,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":22,"num_read":5085,"num_read_kb":33707,"num_write":573,"num_write_kb":19598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":15174176,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":72}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"97'534","reported_seq":5499,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.014639+0000","last_change":"2024-09-11T09:29:55.791461+0000","last_active":"2024-09-11T09:30:23.014639+0000","last_peered":"2024-09-11T09:30:23.014639+0000","last_clean":"2024-09-11T09:30:23.014639+0000","last_became_active":"2024-09-11T09:29:55.788285+0000","last_became_peered":"2024-09-11T09:29:55.788285+0000","last_unstale":"2024-09-11T09:30:23.014639+0000","last_undegraded":"2024-09-11T09:30:23.014639+0000","last_fullsized":"2024-09-11T09:30:23.014639+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:44.508548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001676,"stat_sum":{"num_bytes":41332660,"num_objects":64,"num_object_clones":14,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":20,"num_read":6376,"num_read_kb":41220,"num_write":604,"num_write_kb":15037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"97'592","reported_seq":7248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.261914+0000","last_change":"2024-09-11T09:28:48.709187+0000","last_active":"2024-09-11T09:30:23.261914+0000","last_peered":"2024-09-11T09:30:23.261914+0000","last_clean":"2024-09-11T09:30:23.261914+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:30:23.261914+0000","last_undegraded":"2024-09-11T09:30:23.261914+0000","last_fullsized":"2024-09-11T09:30:23.261914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:22.746348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015577869999999999,"stat_sum":{"num_bytes":46616379,"num_objects":69,"num_object_clones":19,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":32,"num_read":8844,"num_read_kb":43102,"num_write":778,"num_write_kb":13264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"97'580","reported_seq":13263,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.255949+0000","last_change":"2024-09-11T09:28:48.708858+0000","last_active":"2024-09-11T09:30:23.255949+0000","last_peered":"2024-09-11T09:30:23.255949+0000","last_clean":"2024-09-11T09:30:23.255949+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:30:23.255949+0000","last_undegraded":"2024-09-11T09:30:23.255949+0000","last_fullsized":"2024-09-11T09:30:23.255949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:04.767025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013196460000000001,"stat_sum":{"num_bytes":54947672,"num_objects":81,"num_object_clones":26,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":29,"num_read":14611,"num_read_kb":58741,"num_write":688,"num_write_kb":10914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"97'688","reported_seq":9202,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.051587+0000","last_change":"2024-09-11T09:30:13.862734+0000","last_active":"2024-09-11T09:30:23.051587+0000","last_peered":"2024-09-11T09:30:23.051587+0000","last_clean":"2024-09-11T09:30:09.843156+0000","last_became_active":"2024-09-11T09:30:13.862734+0000","last_became_peered":"2024-09-11T09:30:13.862734+0000","last_unstale":"2024-09-11T09:30:23.051587+0000","last_undegraded":"2024-09-11T09:30:13.862004+0000","last_fullsized":"2024-09-11T09:30:13.861934+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:24:05.289980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012800909999999999,"stat_sum":{"num_bytes":36682406,"num_objects":65,"num_object_clones":20,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":65,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":11321,"num_read_kb":35237,"num_write":951,"num_write_kb":17433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":65}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"96'396","reported_seq":8018,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.357649+0000","last_change":"2024-09-11T09:28:48.704277+0000","last_active":"2024-09-11T09:30:23.357649+0000","last_peered":"2024-09-11T09:30:23.357649+0000","last_clean":"2024-09-11T09:30:23.357649+0000","last_became_active":"2024-09-11T09:28:46.704959+0000","last_became_peered":"2024-09-11T09:28:46.704959+0000","last_unstale":"2024-09-11T09:30:23.357649+0000","last_undegraded":"2024-09-11T09:30:23.357649+0000","last_fullsized":"2024-09-11T09:30:23.357649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:30.633799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081867,"stat_sum":{"num_bytes":44116128,"num_objects":65,"num_object_clones":15,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":24,"num_read":8012,"num_read_kb":41006,"num_write":514,"num_write_kb":14586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"97'549","reported_seq":6392,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.357166+0000","last_change":"2024-09-11T09:28:48.699507+0000","last_active":"2024-09-11T09:30:23.357166+0000","last_peered":"2024-09-11T09:30:23.357166+0000","last_clean":"2024-09-11T09:30:23.357166+0000","last_became_active":"2024-09-11T09:28:46.701636+0000","last_became_peered":"2024-09-11T09:28:46.701636+0000","last_unstale":"2024-09-11T09:30:23.357166+0000","last_undegraded":"2024-09-11T09:30:23.357166+0000","last_fullsized":"2024-09-11T09:30:23.357166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:41.714048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058014000000000004,"stat_sum":{"num_bytes":62744299,"num_objects":76,"num_object_clones":25,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":27,"num_read":7972,"num_read_kb":58311,"num_write":619,"num_write_kb":28617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"97'404","reported_seq":2511,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:22.985551+0000","last_change":"2024-09-11T09:30:13.862728+0000","last_active":"2024-09-11T09:30:22.985551+0000","last_peered":"2024-09-11T09:30:22.985551+0000","last_clean":"2024-09-11T09:30:12.072844+0000","last_became_active":"2024-09-11T09:30:13.862728+0000","last_became_peered":"2024-09-11T09:30:13.862728+0000","last_unstale":"2024-09-11T09:30:22.985551+0000","last_undegraded":"2024-09-11T09:30:13.862143+0000","last_fullsized":"2024-09-11T09:30:13.862047+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:49.103237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20723252,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":3495,"num_read_kb":23966,"num_write":435,"num_write_kb":7271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":2421812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":54}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"97'350","reported_seq":5063,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.357594+0000","last_change":"2024-09-11T09:30:13.866595+0000","last_active":"2024-09-11T09:30:23.357594+0000","last_peered":"2024-09-11T09:30:23.357594+0000","last_clean":"2024-09-11T09:30:10.657669+0000","last_became_active":"2024-09-11T09:30:13.866595+0000","last_became_peered":"2024-09-11T09:30:13.866595+0000","last_unstale":"2024-09-11T09:30:23.357594+0000","last_undegraded":"2024-09-11T09:30:13.865196+0000","last_fullsized":"2024-09-11T09:30:13.864152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:49.052327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001366531,"stat_sum":{"num_bytes":36708549,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":19,"num_read":5045,"num_read_kb":30130,"num_write":460,"num_write_kb":11112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":56}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"97'918","reported_seq":8247,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.993995+0000","last_change":"2024-09-11T09:29:43.693191+0000","last_active":"2024-09-11T09:30:22.993995+0000","last_peered":"2024-09-11T09:30:22.993995+0000","last_clean":"2024-09-11T09:30:22.993995+0000","last_became_active":"2024-09-11T09:29:43.692006+0000","last_became_peered":"2024-09-11T09:29:43.692006+0000","last_unstale":"2024-09-11T09:30:22.993995+0000","last_undegraded":"2024-09-11T09:30:22.993995+0000","last_fullsized":"2024-09-11T09:30:22.993995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:06.794458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017410150000000001,"stat_sum":{"num_bytes":24656190,"num_objects":78,"num_object_clones":24,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":31,"num_read":11934,"num_read_kb":40475,"num_write":1241,"num_write_kb":9469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2540364,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"97'506","reported_seq":5660,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:22.985603+0000","last_change":"2024-09-11T09:30:13.865816+0000","last_active":"2024-09-11T09:30:22.985603+0000","last_peered":"2024-09-11T09:30:22.985603+0000","last_clean":"2024-09-11T09:30:11.857140+0000","last_became_active":"2024-09-11T09:30:13.865816+0000","last_became_peered":"2024-09-11T09:30:13.865816+0000","last_unstale":"2024-09-11T09:30:22.985603+0000","last_undegraded":"2024-09-11T09:30:13.864937+0000","last_fullsized":"2024-09-11T09:30:13.864859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:18.652510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014058110000000001,"stat_sum":{"num_bytes":57995892,"num_objects":58,"num_object_clones":17,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":58,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":6235,"num_read_kb":34504,"num_write":493,"num_write_kb":13478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":2211490,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":58}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"96'551","reported_seq":7475,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.356165+0000","last_change":"2024-09-11T09:29:42.692169+0000","last_active":"2024-09-11T09:30:23.356165+0000","last_peered":"2024-09-11T09:30:23.356165+0000","last_clean":"2024-09-11T09:30:23.356165+0000","last_became_active":"2024-09-11T09:29:42.690440+0000","last_became_peered":"2024-09-11T09:29:42.690440+0000","last_unstale":"2024-09-11T09:30:23.356165+0000","last_undegraded":"2024-09-11T09:30:23.356165+0000","last_fullsized":"2024-09-11T09:30:23.356165+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:25.526690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063234499999999998,"stat_sum":{"num_bytes":25178746,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":8058,"num_read_kb":42504,"num_write":680,"num_write_kb":9907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":2083870,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":78,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901638+0000","last_change":"2024-09-11T09:29:39.726602+0000","last_active":"2024-09-11T09:30:22.901638+0000","last_peered":"2024-09-11T09:30:22.901638+0000","last_clean":"2024-09-11T09:30:22.901638+0000","last_became_active":"2024-09-11T09:29:39.726294+0000","last_became_peered":"2024-09-11T09:29:39.726294+0000","last_unstale":"2024-09-11T09:30:22.901638+0000","last_undegraded":"2024-09-11T09:30:22.901638+0000","last_fullsized":"2024-09-11T09:30:22.901638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:11.095120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"96'321","reported_seq":2963,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:22.901320+0000","last_change":"2024-09-11T09:30:13.862727+0000","last_active":"2024-09-11T09:30:22.901320+0000","last_peered":"2024-09-11T09:30:22.901320+0000","last_clean":"2024-09-11T09:30:11.974393+0000","last_became_active":"2024-09-11T09:30:13.862727+0000","last_became_peered":"2024-09-11T09:30:13.862727+0000","last_unstale":"2024-09-11T09:30:22.901320+0000","last_undegraded":"2024-09-11T09:30:13.861944+0000","last_fullsized":"2024-09-11T09:30:13.861881+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:33:40.205027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051448599999999998,"stat_sum":{"num_bytes":32290986,"num_objects":56,"num_object_clones":18,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":56,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":21,"num_read":2905,"num_read_kb":18196,"num_write":400,"num_write_kb":7767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":185,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901479+0000","last_change":"2024-09-11T09:29:39.726217+0000","last_active":"2024-09-11T09:30:22.901479+0000","last_peered":"2024-09-11T09:30:22.901479+0000","last_clean":"2024-09-11T09:30:22.901479+0000","last_became_active":"2024-09-11T09:29:39.725965+0000","last_became_peered":"2024-09-11T09:29:39.725965+0000","last_unstale":"2024-09-11T09:30:22.901479+0000","last_undegraded":"2024-09-11T09:30:22.901479+0000","last_fullsized":"2024-09-11T09:30:22.901479+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:26.308300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"97'331","reported_seq":6845,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.407949+0000","last_change":"2024-09-11T09:29:54.786279+0000","last_active":"2024-09-11T09:30:23.407949+0000","last_peered":"2024-09-11T09:30:23.407949+0000","last_clean":"2024-09-11T09:30:23.407949+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:30:23.407949+0000","last_undegraded":"2024-09-11T09:30:23.407949+0000","last_fullsized":"2024-09-11T09:30:23.407949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:35.321759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858231,"num_objects":60,"num_object_clones":17,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":6895,"num_read_kb":31207,"num_write":331,"num_write_kb":12656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":97,"state":"active+undersized","last_fresh":"2024-09-11T09:30:22.901362+0000","last_change":"2024-09-11T09:30:13.865604+0000","last_active":"2024-09-11T09:30:22.901362+0000","last_peered":"2024-09-11T09:30:22.901362+0000","last_clean":"2024-09-11T09:30:11.856992+0000","last_became_active":"2024-09-11T09:30:13.865604+0000","last_became_peered":"2024-09-11T09:30:13.865604+0000","last_unstale":"2024-09-11T09:30:22.901362+0000","last_undegraded":"2024-09-11T09:30:22.901362+0000","last_fullsized":"2024-09-11T09:30:13.864834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:23.935001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"97'512","reported_seq":5950,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901712+0000","last_change":"2024-09-11T09:29:55.788426+0000","last_active":"2024-09-11T09:30:22.901712+0000","last_peered":"2024-09-11T09:30:22.901712+0000","last_clean":"2024-09-11T09:30:22.901712+0000","last_became_active":"2024-09-11T09:29:55.787180+0000","last_became_peered":"2024-09-11T09:29:55.787180+0000","last_unstale":"2024-09-11T09:30:22.901712+0000","last_undegraded":"2024-09-11T09:30:22.901712+0000","last_fullsized":"2024-09-11T09:30:22.901712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:12:31.699993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000729355,"stat_sum":{"num_bytes":35769916,"num_objects":71,"num_object_clones":21,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":29,"num_read":5845,"num_read_kb":34154,"num_write":542,"num_write_kb":16989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":97,"state":"active+undersized","last_fresh":"2024-09-11T09:30:22.901351+0000","last_change":"2024-09-11T09:30:13.862684+0000","last_active":"2024-09-11T09:30:22.901351+0000","last_peered":"2024-09-11T09:30:22.901351+0000","last_clean":"2024-09-11T09:28:35.443489+0000","last_became_active":"2024-09-11T09:30:13.862684+0000","last_became_peered":"2024-09-11T09:30:13.862684+0000","last_unstale":"2024-09-11T09:30:22.901351+0000","last_undegraded":"2024-09-11T09:30:22.901351+0000","last_fullsized":"2024-09-11T09:30:13.861822+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:55.133591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"97'420","reported_seq":5182,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.356516+0000","last_change":"2024-09-11T09:29:54.792764+0000","last_active":"2024-09-11T09:30:23.356516+0000","last_peered":"2024-09-11T09:30:23.356516+0000","last_clean":"2024-09-11T09:30:23.356516+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:30:23.356516+0000","last_undegraded":"2024-09-11T09:30:23.356516+0000","last_fullsized":"2024-09-11T09:30:23.356516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:40:00.506585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016227870000000001,"stat_sum":{"num_bytes":26328710,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":26,"num_read":5554,"num_read_kb":45644,"num_write":432,"num_write_kb":7873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"97'580","reported_seq":12591,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.357386+0000","last_change":"2024-09-11T09:30:13.862736+0000","last_active":"2024-09-11T09:30:23.357386+0000","last_peered":"2024-09-11T09:30:23.357386+0000","last_clean":"2024-09-11T09:30:11.865833+0000","last_became_active":"2024-09-11T09:30:13.862736+0000","last_became_peered":"2024-09-11T09:30:13.862736+0000","last_unstale":"2024-09-11T09:30:23.357386+0000","last_undegraded":"2024-09-11T09:30:13.861981+0000","last_fullsized":"2024-09-11T09:30:13.861812+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:29:00.526748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077809799999999996,"stat_sum":{"num_bytes":34158096,"num_objects":69,"num_object_clones":23,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":69,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":13887,"num_read_kb":41725,"num_write":460,"num_write_kb":8265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":69}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":177,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.903734+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:30:22.903734+0000","last_peered":"2024-09-11T09:30:22.903734+0000","last_clean":"2024-09-11T09:30:22.903734+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:30:22.903734+0000","last_undegraded":"2024-09-11T09:30:22.903734+0000","last_fullsized":"2024-09-11T09:30:22.903734+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"96'1078","reported_seq":10987,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901285+0000","last_change":"2024-09-11T09:29:55.788949+0000","last_active":"2024-09-11T09:30:22.901285+0000","last_peered":"2024-09-11T09:30:22.901285+0000","last_clean":"2024-09-11T09:30:22.901285+0000","last_became_active":"2024-09-11T09:29:55.787552+0000","last_became_peered":"2024-09-11T09:29:55.787552+0000","last_unstale":"2024-09-11T09:30:22.901285+0000","last_undegraded":"2024-09-11T09:30:22.901285+0000","last_fullsized":"2024-09-11T09:30:22.901285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1078,"log_dups_size":0,"ondisk_log_size":1078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:45:52.896689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057083399999999999,"stat_sum":{"num_bytes":58025577,"num_objects":77,"num_object_clones":22,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":25,"num_read":14221,"num_read_kb":58904,"num_write":1679,"num_write_kb":18353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":58,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901497+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:30:22.901497+0000","last_peered":"2024-09-11T09:30:22.901497+0000","last_clean":"2024-09-11T09:30:22.901497+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:30:22.901497+0000","last_undegraded":"2024-09-11T09:30:22.901497+0000","last_fullsized":"2024-09-11T09:30:22.901497+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"97'386","reported_seq":9228,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:22.990140+0000","last_change":"2024-09-11T09:30:13.862372+0000","last_active":"2024-09-11T09:30:22.990140+0000","last_peered":"2024-09-11T09:30:22.990140+0000","last_clean":"2024-09-11T09:30:09.955291+0000","last_became_active":"2024-09-11T09:30:13.862372+0000","last_became_peered":"2024-09-11T09:30:13.862372+0000","last_unstale":"2024-09-11T09:30:22.990140+0000","last_undegraded":"2024-09-11T09:30:13.861872+0000","last_fullsized":"2024-09-11T09:30:13.861767+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:08.455186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481087,"stat_sum":{"num_bytes":51046051,"num_objects":69,"num_object_clones":20,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":69,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":28,"num_read":10801,"num_read_kb":68583,"num_write":316,"num_write_kb":7137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":69}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901099+0000","last_change":"2024-09-11T09:29:39.724370+0000","last_active":"2024-09-11T09:30:22.901099+0000","last_peered":"2024-09-11T09:30:22.901099+0000","last_clean":"2024-09-11T09:30:22.901099+0000","last_became_active":"2024-09-11T09:29:39.724057+0000","last_became_peered":"2024-09-11T09:29:39.724057+0000","last_unstale":"2024-09-11T09:30:22.901099+0000","last_undegraded":"2024-09-11T09:30:22.901099+0000","last_fullsized":"2024-09-11T09:30:22.901099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:08.072784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"97'338","reported_seq":4398,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.260879+0000","last_change":"2024-09-11T09:28:48.699619+0000","last_active":"2024-09-11T09:30:23.260879+0000","last_peered":"2024-09-11T09:30:23.260879+0000","last_clean":"2024-09-11T09:30:23.260879+0000","last_became_active":"2024-09-11T09:28:46.698963+0000","last_became_peered":"2024-09-11T09:28:46.698963+0000","last_unstale":"2024-09-11T09:30:23.260879+0000","last_undegraded":"2024-09-11T09:30:23.260879+0000","last_fullsized":"2024-09-11T09:30:23.260879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:49.038322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055240599999999997,"stat_sum":{"num_bytes":39889058,"num_objects":75,"num_object_clones":17,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":33,"num_read":4123,"num_read_kb":40913,"num_write":362,"num_write_kb":20403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":194,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.903732+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:30:22.903732+0000","last_peered":"2024-09-11T09:30:22.903732+0000","last_clean":"2024-09-11T09:30:22.903732+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:30:22.903732+0000","last_undegraded":"2024-09-11T09:30:22.903732+0000","last_fullsized":"2024-09-11T09:30:22.903732+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":269,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:22.901110+0000","last_change":"2024-09-11T09:30:13.865591+0000","last_active":"2024-09-11T09:30:22.901110+0000","last_peered":"2024-09-11T09:30:22.901110+0000","last_clean":"2024-09-11T09:30:11.856534+0000","last_became_active":"2024-09-11T09:30:13.865591+0000","last_became_peered":"2024-09-11T09:30:13.865591+0000","last_unstale":"2024-09-11T09:30:22.901110+0000","last_undegraded":"2024-09-11T09:30:13.864233+0000","last_fullsized":"2024-09-11T09:30:13.864163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:45.077238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"96'1449","reported_seq":12422,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.356237+0000","last_change":"2024-09-11T09:29:55.789602+0000","last_active":"2024-09-11T09:30:23.356237+0000","last_peered":"2024-09-11T09:30:23.356237+0000","last_clean":"2024-09-11T09:30:23.356237+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:30:23.356237+0000","last_undegraded":"2024-09-11T09:30:23.356237+0000","last_fullsized":"2024-09-11T09:30:23.356237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1449,"log_dups_size":0,"ondisk_log_size":1449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:42.517657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61633918,"num_objects":93,"num_object_clones":28,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":37,"num_read":17472,"num_read_kb":62185,"num_write":2097,"num_write_kb":15098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"97'506","reported_seq":8395,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.985216+0000","last_change":"2024-09-11T09:29:54.791157+0000","last_active":"2024-09-11T09:30:22.985216+0000","last_peered":"2024-09-11T09:30:22.985216+0000","last_clean":"2024-09-11T09:30:22.985216+0000","last_became_active":"2024-09-11T09:29:54.790054+0000","last_became_peered":"2024-09-11T09:29:54.790054+0000","last_unstale":"2024-09-11T09:30:22.985216+0000","last_undegraded":"2024-09-11T09:30:22.985216+0000","last_fullsized":"2024-09-11T09:30:22.985216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:08:07.658943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042413100000000001,"stat_sum":{"num_bytes":47619803,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":10070,"num_read_kb":45799,"num_write":553,"num_write_kb":13001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":1628771,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"97'690","reported_seq":7055,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.901618+0000","last_change":"2024-09-11T09:29:42.692077+0000","last_active":"2024-09-11T09:30:22.901618+0000","last_peered":"2024-09-11T09:30:22.901618+0000","last_clean":"2024-09-11T09:30:22.901618+0000","last_became_active":"2024-09-11T09:29:42.689141+0000","last_became_peered":"2024-09-11T09:29:42.689141+0000","last_unstale":"2024-09-11T09:30:22.901618+0000","last_undegraded":"2024-09-11T09:30:22.901618+0000","last_fullsized":"2024-09-11T09:30:22.901618+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:28.072546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001003117,"stat_sum":{"num_bytes":30566172,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":14,"num_read":8110,"num_read_kb":44621,"num_write":1039,"num_write_kb":9084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":3514193,"num_keys_recovered":31,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"97'278","reported_seq":7629,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:22.982267+0000","last_change":"2024-09-11T09:30:13.862489+0000","last_active":"2024-09-11T09:30:22.982267+0000","last_peered":"2024-09-11T09:30:22.982267+0000","last_clean":"2024-09-11T09:30:10.774695+0000","last_became_active":"2024-09-11T09:30:13.862489+0000","last_became_peered":"2024-09-11T09:30:13.862489+0000","last_unstale":"2024-09-11T09:30:22.982267+0000","last_undegraded":"2024-09-11T09:30:13.861859+0000","last_fullsized":"2024-09-11T09:30:13.861756+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:01:17.911274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073583100000000005,"stat_sum":{"num_bytes":24596054,"num_objects":59,"num_object_clones":18,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":59,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":21,"num_read":7472,"num_read_kb":35756,"num_write":293,"num_write_kb":5702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":59}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"97'303","reported_seq":5970,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.025348+0000","last_change":"2024-09-11T09:30:13.866371+0000","last_active":"2024-09-11T09:30:23.025348+0000","last_peered":"2024-09-11T09:30:23.025348+0000","last_clean":"2024-09-11T09:30:11.862075+0000","last_became_active":"2024-09-11T09:30:13.866371+0000","last_became_peered":"2024-09-11T09:30:13.866371+0000","last_unstale":"2024-09-11T09:30:23.025348+0000","last_undegraded":"2024-09-11T09:30:13.864501+0000","last_fullsized":"2024-09-11T09:30:13.864415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:21:59.112511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001413711,"stat_sum":{"num_bytes":31464502,"num_objects":57,"num_object_clones":14,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":57,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":27,"num_read":6365,"num_read_kb":28260,"num_write":286,"num_write_kb":9485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":57}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"97'476","reported_seq":5527,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.037876+0000","last_change":"2024-09-11T09:29:54.790570+0000","last_active":"2024-09-11T09:30:23.037876+0000","last_peered":"2024-09-11T09:30:23.037876+0000","last_clean":"2024-09-11T09:30:23.037876+0000","last_became_active":"2024-09-11T09:29:54.789371+0000","last_became_peered":"2024-09-11T09:29:54.789371+0000","last_unstale":"2024-09-11T09:30:23.037876+0000","last_undegraded":"2024-09-11T09:30:23.037876+0000","last_fullsized":"2024-09-11T09:30:23.037876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:41.409566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086609299999999996,"stat_sum":{"num_bytes":66627644,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":38,"num_read":5723,"num_read_kb":45978,"num_write":453,"num_write_kb":13231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"97'426","reported_seq":6102,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.994214+0000","last_change":"2024-09-11T09:28:48.709547+0000","last_active":"2024-09-11T09:30:22.994214+0000","last_peered":"2024-09-11T09:30:22.994214+0000","last_clean":"2024-09-11T09:30:22.994214+0000","last_became_active":"2024-09-11T09:28:46.701376+0000","last_became_peered":"2024-09-11T09:28:46.701376+0000","last_unstale":"2024-09-11T09:30:22.994214+0000","last_undegraded":"2024-09-11T09:30:22.994214+0000","last_fullsized":"2024-09-11T09:30:22.994214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:24.893954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017071,"stat_sum":{"num_bytes":35663947,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":24,"num_read":7717,"num_read_kb":37955,"num_write":387,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"97'337","reported_seq":12293,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.903951+0000","last_change":"2024-09-11T09:29:42.694495+0000","last_active":"2024-09-11T09:30:22.903951+0000","last_peered":"2024-09-11T09:30:22.903951+0000","last_clean":"2024-09-11T09:30:22.903951+0000","last_became_active":"2024-09-11T09:29:42.693576+0000","last_became_peered":"2024-09-11T09:29:42.693576+0000","last_unstale":"2024-09-11T09:30:22.903951+0000","last_undegraded":"2024-09-11T09:30:22.903951+0000","last_fullsized":"2024-09-11T09:30:22.903951+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:47.702105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001864426,"stat_sum":{"num_bytes":62306901,"num_objects":57,"num_object_clones":17,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":20,"num_read":12530,"num_read_kb":56118,"num_write":400,"num_write_kb":18058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"97'688","reported_seq":11935,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.357370+0000","last_change":"2024-09-11T09:30:13.864660+0000","last_active":"2024-09-11T09:30:23.357370+0000","last_peered":"2024-09-11T09:30:23.357370+0000","last_clean":"2024-09-11T09:30:11.966889+0000","last_became_active":"2024-09-11T09:30:13.864660+0000","last_became_peered":"2024-09-11T09:30:13.864660+0000","last_unstale":"2024-09-11T09:30:23.357370+0000","last_undegraded":"2024-09-11T09:30:13.864000+0000","last_fullsized":"2024-09-11T09:30:13.863915+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:41.123081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049006899999999999,"stat_sum":{"num_bytes":33004687,"num_objects":68,"num_object_clones":23,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":23,"num_read":13310,"num_read_kb":53019,"num_write":854,"num_write_kb":13036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":68}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"97'357","reported_seq":4263,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:23.357388+0000","last_change":"2024-09-11T09:29:42.691293+0000","last_active":"2024-09-11T09:30:23.357388+0000","last_peered":"2024-09-11T09:30:23.357388+0000","last_clean":"2024-09-11T09:30:23.357388+0000","last_became_active":"2024-09-11T09:29:42.689937+0000","last_became_peered":"2024-09-11T09:29:42.689937+0000","last_unstale":"2024-09-11T09:30:23.357388+0000","last_undegraded":"2024-09-11T09:30:23.357388+0000","last_fullsized":"2024-09-11T09:30:23.357388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:00:36.901774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067916600000000004,"stat_sum":{"num_bytes":46539815,"num_objects":63,"num_object_clones":22,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":27,"num_read":3651,"num_read_kb":25453,"num_write":395,"num_write_kb":22366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"97'645","reported_seq":6957,"reported_epoch":97,"state":"active+undersized+degraded","last_fresh":"2024-09-11T09:30:23.432059+0000","last_change":"2024-09-11T09:30:13.865607+0000","last_active":"2024-09-11T09:30:23.432059+0000","last_peered":"2024-09-11T09:30:23.432059+0000","last_clean":"2024-09-11T09:30:09.945221+0000","last_became_active":"2024-09-11T09:30:13.865607+0000","last_became_peered":"2024-09-11T09:30:13.865607+0000","last_unstale":"2024-09-11T09:30:23.432059+0000","last_undegraded":"2024-09-11T09:30:13.864543+0000","last_fullsized":"2024-09-11T09:30:13.864471+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:07:45.876087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054829999999999,"stat_sum":{"num_bytes":24978096,"num_objects":76,"num_object_clones":19,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":76,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":31,"num_read":9590,"num_read_kb":29313,"num_write":646,"num_write_kb":5683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":6102690,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":76}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"97'596","reported_seq":6560,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-11T09:30:22.984567+0000","last_change":"2024-09-11T09:29:42.691283+0000","last_active":"2024-09-11T09:30:22.984567+0000","last_peered":"2024-09-11T09:30:22.984567+0000","last_clean":"2024-09-11T09:30:22.984567+0000","last_became_active":"2024-09-11T09:29:42.689610+0000","last_became_peered":"2024-09-11T09:29:42.689610+0000","last_unstale":"2024-09-11T09:30:22.984567+0000","last_undegraded":"2024-09-11T09:30:22.984567+0000","last_fullsized":"2024-09-11T09:30:22.984567+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:34.988738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085759499999999995,"stat_sum":{"num_bytes":28380524,"num_objects":76,"num_object_clones":20,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":29,"num_read":8174,"num_read_kb":19901,"num_write":714,"num_write_kb":9791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":1113070,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1287963545,"num_objects":2119,"num_object_clones":604,"num_object_copies":4238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":759,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2119,"num_whiteouts":827,"num_read":271403,"num_read_kb":1306587,"num_write":20675,"num_write_kb":415150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2817,"num_bytes_recovered":125194288,"num_keys_recovered":360,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56213504,"data_stored":461157973,"data_compressed":19804831,"data_compressed_allocated":37376000,"data_compressed_original":445535744,"omap_allocated":0,"internal_metadata":0},"log_size":17303,"ondisk_log_size":17303,"up":52,"acting":52,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":14,"acting":14,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":1,"acting":1,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607578,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85128,"kb_used_data":9524,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94286712,"statfs":{"total":96636764160,"available":96549593088,"internally_reserved":0,"allocated":9752576,"data_stored":63698174,"data_compressed":3006519,"data_compressed_allocated":5251072,"data_compressed_original":60511232,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607578,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88184,"kb_used_data":12584,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94283656,"statfs":{"total":96636764160,"available":96546463744,"internally_reserved":0,"allocated":12886016,"data_stored":89389026,"data_compressed":4185605,"data_compressed_allocated":7868416,"data_compressed_original":85738496,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607578,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68000,"kb_used_data":10448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94303840,"statfs":{"total":96636764160,"available":96567132160,"internally_reserved":0,"allocated":10698752,"data_stored":88729138,"data_compressed":3186290,"data_compressed_allocated":6561792,"data_compressed_original":85886976,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607579,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53364,"kb_used_data":9696,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94318476,"statfs":{"total":96636764160,"available":96582119424,"internally_reserved":0,"allocated":9928704,"data_stored":73449444,"data_compressed":3388544,"data_compressed_allocated":6189056,"data_compressed_original":70747136,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607579,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89516,"kb_used_data":9752,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94282324,"statfs":{"total":96636764160,"available":96545099776,"internally_reserved":0,"allocated":9986048,"data_stored":71043487,"data_compressed":3544987,"data_compressed_allocated":5861376,"data_compressed_original":67957760,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640279,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48568,"kb_used_data":9056,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323272,"statfs":{"total":96636764160,"available":96587030528,"internally_reserved":0,"allocated":9273344,"data_stored":77054096,"data_compressed":2306203,"data_compressed_allocated":5394432,"data_compressed_original":74268160,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8187904,"data_stored":76045578,"data_compressed":2297528,"data_compressed_allocated":5357568,"data_compressed_original":73694720,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8933376,"data_stored":70829511,"data_compressed":3604641,"data_compressed_allocated":5939200,"data_compressed_original":68203520,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8974336,"data_stored":73551391,"data_compressed":3479537,"data_compressed_allocated":6316032,"data_compressed_original":71267328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9580544,"data_stored":87924860,"data_compressed":3207769,"data_compressed_allocated":6569984,"data_compressed_original":85542912,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11894784,"data_stored":89327706,"data_compressed":4197424,"data_compressed_allocated":7909376,"data_compressed_original":86070272,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8642560,"data_stored":63478927,"data_compressed":3017932,"data_compressed_allocated":5283840,"data_compressed_original":60756992,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:30:27.143 INFO:tasks.thrashosds.thrasher:Setting 3.18 to [0, 1, 5, 3] 2024-09-11T09:30:27.143 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.18', '0', '1', '5', '3'] 2024-09-11T09:30:27.144 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.18 0 1 5 3 2024-09-11T09:30:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:5918 write 0xe186e00 thru 0xe19033e (0x953f bytes) 2024-09-11T09:30:27.144 INFO:teuthology.orchestra.run.smithi107.stdout:5919 read 0x94ef600 thru 0x94fd4f2 (0xdef3 bytes) 2024-09-11T09:30:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:5920 punch from 0x1f92600 to 0x1f92a00, (0x400 bytes) 2024-09-11T09:30:27.145 INFO:teuthology.orchestra.run.smithi107.stdout:5921 write 0xbfd4600 thru 0xbfde411 (0x9e12 bytes) 2024-09-11T09:30:27.149 INFO:teuthology.orchestra.run.smithi107.stdout:5922 trunc from 0xe190400 to 0xa637800 2024-09-11T09:30:27.170 INFO:teuthology.orchestra.run.smithi107.stdout:5927 punch from 0x4ded400 to 0x4dfaa00, (0xd600 bytes) 2024-09-11T09:30:27.172 INFO:teuthology.orchestra.run.smithi107.stdout:5929 read 0x89de400 thru 0x89e5272 (0x6e73 bytes) 2024-09-11T09:30:27.172 INFO:teuthology.orchestra.run.smithi107.stdout:5930 write 0x3c7ae00 thru 0x3c83b69 (0x8d6a bytes) 2024-09-11T09:30:27.174 INFO:teuthology.orchestra.run.smithi107.stdout:5934 trunc from 0xa637800 to 0xb767e00 2024-09-11T09:30:27.183 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:27.183+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:27.187 INFO:teuthology.orchestra.run.smithi107.stdout:5935 read 0x63f200 thru 0x64b7f3 (0xc5f4 bytes) 2024-09-11T09:30:27.189 INFO:teuthology.orchestra.run.smithi107.stdout:5937 read 0xa889400 thru 0xa88cf57 (0x3b58 bytes) 2024-09-11T09:30:27.189 INFO:teuthology.orchestra.run.smithi107.stdout:5938 read 0x1905a00 thru 0x1910ecb (0xb4cc bytes) 2024-09-11T09:30:27.189 INFO:teuthology.orchestra.run.smithi107.stdout:5939 write 0x39ce200 thru 0x39d37cc (0x55cd bytes) 2024-09-11T09:30:27.203 INFO:teuthology.orchestra.run.smithi107.stdout:5940 write 0xd5f2000 thru 0xd5fea1b (0xca1c bytes) 2024-09-11T09:30:27.203 INFO:teuthology.orchestra.run.smithi107.stdout:5941 read 0x6bc400 thru 0x6c6021 (0x9c22 bytes) 2024-09-11T09:30:27.204 INFO:teuthology.orchestra.run.smithi107.stdout:5943 punch from 0xa8b3600 to 0xa8be600, (0xb000 bytes) 2024-09-11T09:30:27.216 INFO:teuthology.orchestra.run.smithi107.stdout:5944 trunc from 0xd5fec00 to 0x463f800 2024-09-11T09:30:27.236 INFO:teuthology.orchestra.run.smithi107.stdout:5946 punch from 0x34b9400 to 0x34c8600, (0xf200 bytes) 2024-09-11T09:30:27.238 INFO:teuthology.orchestra.run.smithi107.stdout:5948 read 0x124ea00 thru 0x12525c0 (0x3bc1 bytes) 2024-09-11T09:30:27.238 INFO:teuthology.orchestra.run.smithi107.stdout:5953 read 0x10ff400 thru 0x1101019 (0x1c1a bytes) 2024-09-11T09:30:27.255 INFO:teuthology.orchestra.run.smithi107.stdout:5955 write 0xaee8200 thru 0xaeeff13 (0x7d14 bytes) 2024-09-11T09:30:27.256 INFO:teuthology.orchestra.run.smithi107.stdout:5957 write 0x5bee200 thru 0x5bf083f (0x2640 bytes) 2024-09-11T09:30:27.256 INFO:teuthology.orchestra.run.smithi107.stdout:5958 punch from 0x3497600 to 0x34a0e00, (0x9800 bytes) 2024-09-11T09:30:27.257 INFO:teuthology.orchestra.run.smithi107.stdout:5960 read 0x51d9600 thru 0x51dd2b7 (0x3cb8 bytes) 2024-09-11T09:30:27.257 INFO:teuthology.orchestra.run.smithi107.stdout:5961 read 0x301ce00 thru 0x3027882 (0xaa83 bytes) 2024-09-11T09:30:27.266 INFO:teuthology.orchestra.run.smithi107.stdout:5964 trunc from 0xaef0000 to 0x5037200 2024-09-11T09:30:27.267 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:27.268+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:27.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:27.280 INFO:teuthology.orchestra.run.smithi107.stdout:5965 read 0x2a61000 thru 0x2a696e0 (0x86e1 bytes) 2024-09-11T09:30:27.281 INFO:teuthology.orchestra.run.smithi107.stdout:5966 read 0x2861400 thru 0x2865ae6 (0x46e7 bytes) 2024-09-11T09:30:27.281 INFO:teuthology.orchestra.run.smithi107.stdout:5968 write 0x3b60200 thru 0x3b60672 (0x473 bytes) 2024-09-11T09:30:27.300 INFO:teuthology.orchestra.run.smithi107.stdout:5971 write 0xa230400 thru 0xa23fd4b (0xf94c bytes) 2024-09-11T09:30:27.304 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:27.304 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:27.309 INFO:teuthology.orchestra.run.smithi107.stdout:5974 trunc from 0xa23fe00 to 0xdccc00 2024-09-11T09:30:27.315 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:27.331 INFO:teuthology.orchestra.run.smithi107.stdout:5975 trunc from 0xdccc00 to 0x2021a00 2024-09-11T09:30:27.340 INFO:teuthology.orchestra.run.smithi107.stdout:5977 punch from 0x9db800 to 0x9e1400, (0x5c00 bytes) 2024-09-11T09:30:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:5978 write 0xcaab400 thru 0xcab65a4 (0xb1a5 bytes) 2024-09-11T09:30:27.363 INFO:teuthology.orchestra.run.smithi107.stdout:5979 write 0x451c00 thru 0x453d75 (0x2176 bytes) 2024-09-11T09:30:27.364 INFO:teuthology.orchestra.run.smithi107.stdout:5982 punch from 0x53ef200 to 0x53f8400, (0x9200 bytes) 2024-09-11T09:30:27.365 INFO:teuthology.orchestra.run.smithi107.stdout:5983 write 0x9436600 thru 0x943ab0f (0x4510 bytes) 2024-09-11T09:30:27.366 INFO:teuthology.orchestra.run.smithi107.stdout:5984 read 0xb2bd000 thru 0xb2bf2b4 (0x22b5 bytes) 2024-09-11T09:30:27.366 INFO:teuthology.orchestra.run.smithi107.stdout:5986 write 0x9fa1a00 thru 0x9fa4204 (0x2805 bytes) 2024-09-11T09:30:27.369 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:27.369+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:27.375 INFO:teuthology.orchestra.run.smithi107.stdout:5988 trunc from 0xcab6600 to 0x4785600 2024-09-11T09:30:27.391 INFO:teuthology.orchestra.run.smithi107.stdout:5989 read 0x217d800 thru 0x21823be (0x4bbf bytes) 2024-09-11T09:30:27.392 INFO:teuthology.orchestra.run.smithi107.stdout:5990 write 0x41b2000 thru 0x41b2349 (0x34a bytes) 2024-09-11T09:30:27.392 INFO:teuthology.orchestra.run.smithi107.stdout:5993 read 0x3d62800 thru 0x3d6d178 (0xa979 bytes) 2024-09-11T09:30:27.392 INFO:teuthology.orchestra.run.smithi107.stdout:5995 read 0x1058400 thru 0x1063b07 (0xb708 bytes) 2024-09-11T09:30:27.393 INFO:teuthology.orchestra.run.smithi107.stdout:5996 read 0x28ca800 thru 0x28ca915 (0x116 bytes) 2024-09-11T09:30:27.393 INFO:teuthology.orchestra.run.smithi107.stdout:5998 clone 43 order 19 su 16384 sc 14 2024-09-11T09:30:27.432 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:27.450 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:27.450 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:27.467 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:27.467 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:27.468 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:27.468 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:27.468 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:27.468 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:27.469 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:27.470 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:27.470+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:27.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:27.484 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:27.484 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:27.501 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:27.501 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:27.537 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:27.537 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:27.540 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:27.541 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:27.571 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:27.572+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:27.587 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:27.587 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:27.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:27.587 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:27.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396531517, 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.592 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396510798000001, 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:27.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396490279999998, 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:27.597 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:27.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:27.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.601 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396467622999999, 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.602 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:27.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.606 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396441900999999, 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.607 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:27.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396421815, 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396388289000001, 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.616 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:27.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.620 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396365319000001, 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:27.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:27.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396344929000001, 2024-09-11T09:30:27.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:27.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396324905, 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.630 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:27.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.634 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396305294000001, 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.635 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:27.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.639 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396280260000001, 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.640 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:27.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:27.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:27.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 18.396256343000001, 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:27.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732887277, 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:27.650 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:27.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:27.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732864499000002, 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732836888000001, 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:27.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:27.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732815982999998, 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:27.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:27.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:27.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732795636999999, 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:27.673 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:27.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732611815999999, 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:27.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:27.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.732582060999999, 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:27.684 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:27.689 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:27.691 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:27.692 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:27.692+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:27.791 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:27.792+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:27.795 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:27.795 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:27.795 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:27.795 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:27.804 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:27.892 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:27.893+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:27.948 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:27.948 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:27.948 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:27.957 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:27.962 INFO:teuthology.orchestra.run.smithi017.stderr:set 3.18 pg_upmap_items mapping to [0->1,5->3] 2024-09-11T09:30:27.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:30:28.018 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:28.019+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:28.039 INFO:teuthology.orchestra.run.smithi107.stdout:truncating image image_client.0-clone42 from 0x4785600 (overlap 0x458200) to 0x0 2024-09-11T09:30:28.070 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:28.070 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 117.632433427, 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.071 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:28.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:28.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:28.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:28.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.016527699999997, 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:28.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:28.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.016329999, 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:28.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:28.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:28.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.093 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.016071572999998, 2024-09-11T09:30:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:28.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:28.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:28.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:28.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:28.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:28.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:28.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:28.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:28.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:28.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:28.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:28.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:28.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:28.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:28.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:28.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:28.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:28.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:28.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:28.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:28.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:28.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:28.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:28.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:28.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:28.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:28.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.107 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.107 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:28.108 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.016041688999998, 2024-09-11T09:30:28.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:28.108 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.109 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:28.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.109 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:28.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:28.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:28.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:28.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:28.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:28.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:28.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:28.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:28.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:28.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:28.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:28.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.015508537000002, 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:28.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.014631088000002, 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.014592122000003, 2024-09-11T09:30:28.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:28.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:28.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:28.139 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.014548646000001, 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:28.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.014424028999997, 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:28.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.148 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.013858558999999, 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:28.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:28.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.013827923000001, 2024-09-11T09:30:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:28.157 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.158 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:28.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:28.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:28.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:28.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:28.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:28.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:28.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:28.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:28.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:28.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:28.166 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.166 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.167 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.167 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:28.167 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.011691491000001, 2024-09-11T09:30:28.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:28.168 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:28.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:28.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:28.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:28.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:28.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:28.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:28.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.177 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.010171003000004, 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:28.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:28.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.006097173999997, 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:28.183 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:28.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:28.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.005983380000004, 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:28.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:28.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:28.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:28.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.005898698000003, 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:28.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:28.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.005852707000003, 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.005481408999998, 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.206 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:28.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.005339460000002, 2024-09-11T09:30:28.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:28.214 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.215 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:28.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:28.217 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:28.216+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:28.317 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:28.318+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:28.322 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:28.324 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:28.324 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:28.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:28.418 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:28.419+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:28.473 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:28.474 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:28.474 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:28.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:28.525 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:28.526+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:28.586 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246989641999999, 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:28.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:28.591 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246966423, 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.592 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:28.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:28.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246939262000001, 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:28.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:28.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:28.601 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246918737000001, 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:28.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:28.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:28.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246892303999999, 2024-09-11T09:30:28.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:28.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246864152000001, 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:28.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.615 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246833771999999, 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.616 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:28.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246810698000001, 2024-09-11T09:30:28.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:28.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:28.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:28.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246781337000002, 2024-09-11T09:30:28.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:28.629 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246760111, 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.630 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.634 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246739818000002, 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:28.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.639 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246713771, 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:28.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246692190000001, 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:28.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:28.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246671386999999, 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:28.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246651976999999, 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:28.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246615009999999, 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:28.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:28.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246582608000001, 2024-09-11T09:30:28.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:28.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246560441, 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:28.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246535461000001, 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:28.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 19.246515081999998, 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:28.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:28.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:28.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:28.682 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:28.684 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:28.684 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:28.684+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:28.784 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:28.785+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:28.826 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:28.827 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:28.827 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:28.827 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:28.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:28.886 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:28.886+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:28.939 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:28.939 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:28.939 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:28.939 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:28.939 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:28.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:28.964 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:30:28.992 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:28.993+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:29.052 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.069165472999998, 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.053 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.069119428999997, 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:29.058 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:29.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.069107759000005, 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:29.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067819490000005, 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067783789000003, 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:29.072 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:29.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067761751999996, 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:29.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:29.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067726167000004, 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:29.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:29.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.086 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067665824000002, 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:29.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067617139999996, 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:29.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:29.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067560675999999, 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:29.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067541121000005, 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.101 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:29.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067493357999993, 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:29.105 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:29.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.109 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067379619999997, 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:29.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.114 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067360875999995, 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:29.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067321887000006, 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:29.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.067298780000002, 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:29.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:29.128 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.359222289000002, 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:29.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.996822958999999, 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:29.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:29.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.991571882000002, 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:29.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 41.991311512000003, 2024-09-11T09:30:29.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:29.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:29.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:29.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:29.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:29.154 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:29.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:29.156 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:29.156+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:29.256 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:29.257+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:29.276 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:29.276 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:29.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:29.276 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:29.286 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:29.358 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:29.358+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:29.434 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:29.434 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:29.434 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:29.434 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:29.434 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:29.442 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:29.486 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:29.487+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:29.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.639817833999999, 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:29.553 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.639587262000006, 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.639490058000007, 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:29.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.639255293999994, 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.569 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.571 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.639059278000005, 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.574 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:29.576 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.577 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:29.578 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.638790841000002, 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:29.579 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.580 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:29.581 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:29.582 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.638689091000003, 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.583 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:29.584 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.585 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.587 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.638295434, 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.588 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.589 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.590 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:29.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.638169188999996, 2024-09-11T09:30:29.592 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.593 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.595 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:29.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.597 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.637959807000001, 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.599 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:29.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.601 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:29.602 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.637848857999998, 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.603 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:29.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.637592920000003, 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.607 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.608 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:29.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:29.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.611 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.637440390999998, 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.612 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:29.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.637248026999998, 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.616 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.636917608999994, 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.621 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:29.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.624 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.625 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.636641279000003, 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:29.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.630 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.636587359000004, 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.631 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:29.632 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.633 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.636386790000003, 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.635 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:29.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 78.636031251000006, 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.640 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:29.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 46.856765717999998, 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:29.651 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:29.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:29.654 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:29.654 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:29.653+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:29.665 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:29.666 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:29.683 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:29.684 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:29.701 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:29.701 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:29.718 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:29.718 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:29.735 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:29.735 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:29.738 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:29.739 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:29.753 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:29.754+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:29.773 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:29.773 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:29.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:29.854 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:29.855+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:29.930 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:29.931 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:29.940 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:29.955 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:29.956+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:30.056 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:30.056+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:30.065 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.065 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:30.082 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-11T09:30:30.157 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:30.157+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:30.186 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.186 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:30.214 INFO:teuthology.orchestra.run.smithi107.stdout:5999 write 0xd1a4a00 thru 0xd1ae17a (0x977b bytes) 2024-09-11T09:30:30.218 INFO:teuthology.orchestra.run.smithi107.stdout:6000 trunc from 0xd1ae200 to 0xac6f600 2024-09-11T09:30:30.258 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:30.258+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:30.301 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.301 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.310 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:30.359 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:30.359+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:30.384 INFO:teuthology.orchestra.run.smithi107.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-11T09:30:30.455 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.455 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.459 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:30.460+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:30.466 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:30.560 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:30.561+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:30.612 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.613 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:30.663 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:30.663+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:30.762 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:30.762+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:30.765 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.766 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:30.862 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:30.863+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:30.876 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:30.877 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:30.885 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:30.969 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:30.970+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:30.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:30:31.037 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:31.037 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:31.047 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:31.065 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:31.065+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:31.165 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:31.166+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:31.190 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:31.191 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:31.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:31.285 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:31.285+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:31.344 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:31.344 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:31.353 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:31.393 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:31.394+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:31.461 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:31.461 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:31.467 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:31.468+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:31.469 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:31.569 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:31.569+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:31.616 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:31.617 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:31.625 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:31.669 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:31.670+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:31.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:31.776 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:31.776 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:31.776 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:31.777+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:31.785 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:31.848 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:31.865 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:31.865 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:31.871 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:31.872+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:31.882 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:31.899 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:31.917 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:31.917 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:31.933 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:31.934 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:31.937 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:31.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:31.938 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:31.938 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:31.938 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:31.938 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:31.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:31.972 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:31.973+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:31.974 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:30:32.073 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:32.074+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:32.085 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:32.085 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:32.086 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:32.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:32.086 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:32.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:32.174 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:32.174+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011808468000002, 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:32.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.208 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011787749, 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.209 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.213 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011767231, 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:32.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.218 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011744574000002, 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:32.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:32.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011718852000001, 2024-09-11T09:30:32.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:32.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:32.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:32.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:32.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:32.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:32.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:32.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:32.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.231 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.231 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:32.231 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011698765999999, 2024-09-11T09:30:32.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:32.231 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.232 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.232 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.232 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:32.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.233 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.236 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:32.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:32.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.240 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011665239999999, 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:32.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011642269999999, 2024-09-11T09:30:32.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:32.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:32.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:32.251 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.01162188, 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.254 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011601855999999, 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.256 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:32.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.259 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:32.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:32.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.262 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011582245, 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.264 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:32.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.267 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011557211, 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.268 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.269 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.269 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:32.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.269 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:32.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 23.011533293999999, 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.274 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:32.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:32.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.278 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.348164228000002, 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.279 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:32.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.34814145, 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.285 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:32.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:32.287 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.289 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.348113839, 2024-09-11T09:30:32.290 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.291 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:32.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.292 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.292 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.292 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:32.292 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.293 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.293 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:32.293 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.294 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.294 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:32.294 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:32.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.295 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:32.296 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:32.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.296 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.296 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:32.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:32.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.297 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.297 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:32.297 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:32.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.298 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.298 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:32.298 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:32.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.299 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.299 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:32.299 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:32.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.300 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.301 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:32.301 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:32.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.302 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.302 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:32.302 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.348092934, 2024-09-11T09:30:32.302 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:32.303 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.303 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.303 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.303 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.303 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:32.304 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.306 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.306 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:32.306 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:32.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.307 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.308 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.309 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:32.310 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.348072588000001, 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.311 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.312 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.313 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:32.314 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.315 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:32.316 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.347888767000001, 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:32.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:32.318 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.347859012000001, 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:32.327 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:32.329 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:32.329 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:32.329+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:32.430 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:32.430+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:32.433 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:32.433 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:32.433 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:32.433 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:32.442 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:32.530 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:32.531+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:32.584 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:32.584 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:32.584 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:32.592 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:32.634 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:32.634+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:32.731 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:32.732+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:32.738 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 122.300432371, 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.739 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:32.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.684526644000002, 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.746 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:32.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:32.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:32.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.752 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.684328942999997, 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:32.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:32.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:32.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.684070517000002, 2024-09-11T09:30:32.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:32.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:32.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:32.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.684040633000002, 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:32.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.683507480999999, 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:32.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.774 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:32.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.682630031999999, 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.776 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:32.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.780 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.682591066000001, 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:32.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:32.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:32.785 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.682547589999999, 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.786 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:32.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:32.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.682422973000001, 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.681857503000003, 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:32.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.681826866999998, 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:32.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:32.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.679690434999998, 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:32.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:32.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.678169947000001, 2024-09-11T09:30:32.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:32.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.674096118000001, 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:32.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.673982324000001, 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:32.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.673897642, 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:32.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.673851651, 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:32.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.673480353000002, 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.836 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:32.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:32.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 45.673338403999999, 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:32.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.845 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:32.847 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:32.847 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:32.848+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:32.947 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:32.948+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:32.978 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:30:32.979 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:32.979 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-11T09:30:32.979 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-11T09:30:32.979 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-11T09:30:32.979 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-11T09:30:32.991 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop(osd.1.0:4013 3.19 e105/21 3:9b1f1da6:::rbd_data.1106d8b1f4e5.0000000000000005:2c v 105'570, mlcod=99'567)", 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:32.992541+0000", 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00027724500000000002, 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00030082500000000001, 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:32.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992541+0000", 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992541+0000", 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992542+0000", 2024-09-11T09:30:32.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9299999999999999e-07 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992546+0000", 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7439999999999998e-06 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992560+0000", 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3637e-05 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:32.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992569+0000", 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6710000000000005e-06 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992623+0000", 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3956999999999999e-05 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:32.992641+0000", 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8428999999999999e-05 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:32.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.996 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:32.996 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:30:32.996 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-11T09:30:32.996 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:33.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:33.047 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:33.048+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:33.111 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:33.111 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-11T09:30:33.111 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:33.149 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:33.149+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:33.150 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:33.150 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:33.151 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:33.151 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:33.151 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:33.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:33.249 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:33.250+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:33.263 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.924005943000001, 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.264 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:33.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:33.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923982723999998, 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:33.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.273 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923955563, 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.274 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923935038, 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:33.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:33.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:33.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923908605000001, 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:33.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923880452999999, 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:33.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:33.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923850073000001, 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:33.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:33.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923826998999999, 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:33.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:33.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923797638, 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:33.303 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:33.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:33.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923776411999999, 2024-09-11T09:30:33.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:33.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:33.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:33.313 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923756119, 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:33.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923730072000001, 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:33.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:33.322 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923708490999999, 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:33.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.327 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923687688000001, 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:33.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:33.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923668278000001, 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:33.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.336 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923631311000001, 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.337 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:33.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:33.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923598908999999, 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.342 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.343 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:33.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923576742000002, 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:33.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:33.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.351 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923551761999999, 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:33.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 23.923531383, 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:33.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.361 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:33.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:33.363 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:33.363+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:33.464 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:33.464+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:33.507 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:33.507 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:33.507 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:33.507 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:33.515 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:33.565 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:33.566+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:33.659 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:33.659 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:33.659 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:33.659 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:33.659 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:33.666 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:33.667+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:33.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:33.767 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:33.768+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:33.813 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.829624858000003, 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:33.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.829578814000001, 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.818 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:33.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.829567143999995, 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:33.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.824 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.825 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.826 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828278874999995, 2024-09-11T09:30:33.827 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:33.828 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.829 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.830 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:33.831 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828243173999994, 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.832 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.833 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.834 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:33.835 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.836 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828221137, 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.837 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:33.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.841 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828185551999994, 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:33.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.845 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828125209000007, 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:33.846 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:33.850 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828076525, 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.851 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:33.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:33.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828020061000004, 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.856 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:33.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:33.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:33.860 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.828000505999995, 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.827952742999997, 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:33.865 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:33.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.869 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.827839005000001, 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.870 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:33.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.874 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.827820260999999, 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:33.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.827781271999996, 2024-09-11T09:30:33.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:33.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:33.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.827758165000006, 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.884 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.119681674000006, 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:33.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:33.893 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.757282344000004, 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.894 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:33.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:33.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:33.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.752031267, 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:33.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.751770897, 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:33.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:33.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:33.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:33.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:33.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:33.911 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:33.912 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:33.913+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:33.913 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:33.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:33.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:30:34.012 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:34.013+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:34.053 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:34.053 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:34.054 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:34.054 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:34.066 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:34.089 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:34.113 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:34.113+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:34.117 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:34.117 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:34.178 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.178 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.187 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:34.214 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:34.215+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:34.315 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:34.316+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:34.335 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.335 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.336 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:34.336 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:34.344 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:34.352 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:34.353 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:34.369 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:34.370 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:34.386 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:34.387 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:34.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:34.392 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:34.416 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:34.417+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:34.454 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.454 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:34.517 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:34.518+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:34.569 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.570 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:34.617 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:34.619+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:34.685 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.685 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:34.718 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:34.719+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:34.802 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.802 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.811 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:34.819 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:34.820+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:34.912 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:34.912 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:34.920 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:34.920+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:34.921 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:34.999 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:30:35.031 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.031 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:35.056 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:35.057+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:35.120 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:35.121+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:35.151 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.151 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:35.221 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:35.222+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:35.262 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.262 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.271 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:35.322 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:35.323+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:35.414 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.414 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:35.424 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:35.424+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:35.525 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:35.526+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:35.526 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.526 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.535 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:35.626 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:35.627+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:35.636 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.636 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.645 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:35.729 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:35.730+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:35.789 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.789 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.798 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:35.839 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:35.839+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:35.919 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:35.919 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:35.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:35.929 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:35.931+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:36.030 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:36.031+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:36.045 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:36.045 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:36.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:36.132 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:36.132+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:36.166 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:36.166 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:36.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:36.259 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:36.260+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:36.322 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:36.323 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:36.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:36.374 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:36.374+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:36.393 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:36.434 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:36.434+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:36.480 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:36.481 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:36.490 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:36.523 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:36.535 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:36.535+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:36.542 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:36.544 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:36.560 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:36.560 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:36.577 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:36.577 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:36.601 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:36.601 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:36.601 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:36.601 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410339724, 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:36.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.603 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:36.604 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.605 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.605 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:36.605 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.606 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:36.607 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410319005000002, 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.608 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.609 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:36.611 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.612 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410298486999999, 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:36.614 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.615 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.616 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.617 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.41027583, 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.618 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.619 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:36.620 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.621 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:36.622 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410250108, 2024-09-11T09:30:36.623 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:36.623 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.623 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.623 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.623 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.624 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.624 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.624 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:36.624 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.624 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.625 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.626 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.626 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:36.626 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:36.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.626 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.626 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:36.627 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:36.627 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.627 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.627 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.627 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:36.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:36.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.628 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.628 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.628 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:36.628 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:36.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.629 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.629 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:36.629 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:36.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.631 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.631 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:36.631 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:36.631 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.631 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.631 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.632 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.632 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.632 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:36.632 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410230022, 2024-09-11T09:30:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.633 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.634 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.635 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.636 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:36.637 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:36.638 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:36.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410196496000001, 2024-09-11T09:30:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:36.643 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.644 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.645 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:36.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:36.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:36.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:36.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:36.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:36.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:36.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.651 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:36.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:36.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.651 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:36.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:36.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.653 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410173526000001, 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.654 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.656 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:36.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410153136000002, 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.659 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:36.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410133112, 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:36.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.668 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410113501000001, 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:36.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.673 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410088467000001, 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:36.674 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.675 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:36.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 27.410064550000001, 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.682 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:36.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746695484, 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.689 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:36.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746672705999998, 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.694 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:36.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:36.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746645095000002, 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:36.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:36.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:36.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746624189999999, 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:36.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:36.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:36.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746603843999999, 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.711 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:36.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:36.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:36.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:36.716 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746420022999999, 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:36.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.746390267999999, 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:36.722 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:36.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:36.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:36.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:36.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:36.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:36.728 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:36.728 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:36.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:36.730 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:36.730+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:36.796 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:36.796 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:36.814 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:36.815 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:36.817 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:36.817 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:36.829 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:36.830+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:36.837 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:36.837 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:36.838 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:36.838 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:36.846 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:36.931 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:36.931+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:36.989 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:36.989 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:36.990 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:36.998 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:37.015 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:30:37.059 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:37.060+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:37.132 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:37.132+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:37.152 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 126.715114226, 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:37.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:37.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:37.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:37.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:37.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:37.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.099208499, 2024-09-11T09:30:37.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:37.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.169 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.099010798000002, 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.170 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:37.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:37.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.098752372, 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:37.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:37.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.098722488, 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:37.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.098189335999997, 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:37.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:37.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:37.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.097311886999996, 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:37.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:37.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.097272920999998, 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:37.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:37.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:37.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.097229445000004, 2024-09-11T09:30:37.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:37.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:37.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.097104827999999, 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:37.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.096539358000001, 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:37.216 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:37.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:37.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.096508722000003, 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:37.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.094372290000003, 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:37.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:37.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.092851801999998, 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:37.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:37.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.236 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.088777972999999, 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:37.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:37.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:37.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.088664178999998, 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:37.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:37.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.088579496999998, 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:37.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:37.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:37.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:37.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:37.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.088533505999997, 2024-09-11T09:30:37.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:37.257 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:37.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:37.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:37.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.088162208, 2024-09-11T09:30:37.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:37.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 50.088020258999997, 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:37.270 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:37.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:37.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.276 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:37.278 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:37.278 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:37.278+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:37.378 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:37.379+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:37.422 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:37.422 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:37.422 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:37.422 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:37.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:37.514 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:37.515+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:37.574 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:37.574 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:37.574 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:37.574 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:37.574 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:37.579 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:37.580+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:37.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:37.680 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:37.681+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:37.727 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:37.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387754771000001, 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.728 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:37.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:37.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:37.732 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387731551999998, 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:37.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:37.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387704391, 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.737 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:37.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.741 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387683866, 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.742 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387657433000001, 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:37.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387629280999999, 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:37.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387598901, 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:37.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387575826999999, 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.765 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387546466, 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:37.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:37.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387525239999999, 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:37.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:37.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.776 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387504947, 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:37.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.779 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.780 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387478900000001, 2024-09-11T09:30:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.783 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.784 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:37.785 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387457318999999, 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.786 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.787 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:37.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:37.790 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387436516000001, 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387417106000001, 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:37.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387380139000001, 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:37.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.801 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.802 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:37.803 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:37.804 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387347736999999, 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.805 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:37.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:37.807 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.808 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387325570000002, 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.810 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:37.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:37.812 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387300589999999, 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.814 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:37.815 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.816 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.817 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 28.387280211, 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:37.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.819 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:37.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:37.821 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:37.823 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:37.823 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:37.824 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:37.825 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:37.824+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:37.924 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:37.925+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:37.968 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:37.968 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:37.968 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:37.976 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:38.009 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:30:38.024 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:38.025+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:38.122 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:30:38.122 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:38.122 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T09:30:38.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T09:30:38.126 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:38.126+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:38.127 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:38.127 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:38.127 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:38.135 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:38.226 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:38.227+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:38.281 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.298218134999999, 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:38.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.287 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.298172090999998, 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.288 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:38.292 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.298160421000006, 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:38.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.297 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296872152000006, 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:38.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.302 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296836451000004, 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:38.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.306 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296814413999996, 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:38.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296778829000004, 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.312 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:38.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.316 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296718486000003, 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:38.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:38.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296669801999997, 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.323 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:38.324 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:38.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296613338, 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.327 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:38.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.330 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:38.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296593783000006, 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.332 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:38.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:38.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:38.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.340 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:38.341 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296546019999994, 2024-09-11T09:30:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.342 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:38.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:38.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:38.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:38.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.351 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.352 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296432281999998, 2024-09-11T09:30:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:38.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:38.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:38.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296413537999996, 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:38.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:38.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:38.367 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296374549000006, 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.368 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:38.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:38.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.372 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.296351442000002, 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:38.373 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:38.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:38.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:38.377 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.588274951000002, 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.378 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:38.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.381 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:38.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.225875621, 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:38.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.386 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:38.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:38.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.220624544000003, 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:38.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.395 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.220364173999997, 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:38.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:38.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:38.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.400 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:38.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:38.402 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:38.404 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:38.405 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:38.404+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:38.418 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:38.418 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:38.418 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 110, 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "23a8bf95-caba-4fea-a762-61040a1c1b77", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-11T09:28:17.894525+0000", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-11T09:30:38.003875+0000", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2024-09-11T09:30:12.854131+0000", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2024-09-11T09:29:50.757033+0000", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5806080, 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "nodeep-scrub", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "noscrub", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2024-09-11T09:30:38.419 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 5, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 3, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 6, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "luminous", 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "luminous", 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "squid", 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_crimson": false, 2024-09-11T09:30:38.420 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": ".mgr", 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:28.765227+0000", 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:38.421 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:38.422 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "15", 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2024-09-11T09:30:38.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:38.424 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:38.425 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_max": 32, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": {} 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 5.880000114440918, 2024-09-11T09:30:38.426 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 5.880000114440918, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.17000000178813934, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:34.774979+0000", 2024-09-11T09:30:38.427 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2024-09-11T09:30:38.428 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2024-09-11T09:30:38.429 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 2, 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 19, 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:38.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:38.431 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:38.432 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 2.2699999809265137, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 2.2699999809265137, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-11T09:30:38.433 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "pool_client.0", 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:45.573818+0000", 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:38.434 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 32, 2024-09-11T09:30:38.435 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 32, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 32, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 32, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 32, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.436 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "110", 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 45, 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 110, 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 1.7000000476837158, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 1.7000000476837158, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1.4099999666213989, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1.4099999666213989, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "44ce15f5-731e-4d58-9dd7-bd8f186e5783", 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 0, 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 11, 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 90, 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2024-09-11T09:30:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/4208334809", 2024-09-11T09:30:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/4208334809", 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/4208334809", 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/4208334809", 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "exists" 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "dba82e38-a74b-4d42-bad4-9b1e013d4da3", 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 0.4687957763671875, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2024-09-11T09:30:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2024-09-11T09:30:38.450 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6817/361445038", 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6819/361445038", 2024-09-11T09:30:38.452 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6823/361445038", 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6821/361445038", 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "64a33918-9bb2-4d59-a66e-73da15d78cbd", 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:38.453 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 103, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.456 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.457 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/211521567", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/211521567", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/211521567", 2024-09-11T09:30:38.458 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/211521567", 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a4774bc9-54ea-4272-b6b4-a6dd00c4c930", 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:38.459 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6800", 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6801", 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6802", 2024-09-11T09:30:38.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6803", 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6806", 2024-09-11T09:30:38.462 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6807", 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.463 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6804", 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6805", 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6801/2070567113", 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6803/2070567113", 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6807/2070567113", 2024-09-11T09:30:38.464 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6805/2070567113", 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a3984c6d-7250-4e85-a032-1dc0315642d8", 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:38.465 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6816", 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6817", 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6818", 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6819", 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6822", 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6823", 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:38.469 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6820", 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6821", 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.470 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6817/1795828331", 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6819/1795828331", 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6823/1795828331", 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6821/1795828331", 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.471 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "0a40bbb7-9d37-4a1b-b525-86608a3eaaa2", 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 90, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:38.472 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6808", 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6809", 2024-09-11T09:30:38.473 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6810", 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.474 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6811", 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6814", 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6815", 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6812", 2024-09-11T09:30:38.476 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6813", 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6809/3808455039", 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6811/3808455039", 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6815/3808455039", 2024-09-11T09:30:38.477 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6813/3808455039", 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:38.478 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.320025+0000", 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 90 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.922939+0000", 2024-09-11T09:30:38.479 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.876780+0000", 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.854159+0000", 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.081538+0000", 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.482 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.168719+0000", 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [ 2024-09-11T09:30:38.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.18", 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "mappings": [ 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "from": 0, 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "to": 1 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_primaries": [], 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [], 2024-09-11T09:30:38.484 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklist": {}, 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "range_blocklist": {}, 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2024-09-11T09:30:38.485 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [ 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: "snaps": [ 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: "begin": 39, 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: "length": 7 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.486 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [ 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: "snaps": [ 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: "begin": 39, 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: "length": 1 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2024-09-11T09:30:38.487 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {}, 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": { 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_enabled": false, 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_bucket_count": 0, 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "degraded_stretch_mode": 0, 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "recovering_stretch_mode": 0, 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_bucket": 0 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.488 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:38.490 INFO:tasks.thrashosds.thrasher:j is {'epoch': 110, 'fsid': '23a8bf95-caba-4fea-a762-61040a1c1b77', 'created': '2024-09-11T09:28:17.894525+0000', 'modified': '2024-09-11T09:30:38.003875+0000', 'last_up_change': '2024-09-11T09:30:12.854131+0000', 'last_in_change': '2024-09-11T09:29:50.757033+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T09:28:28.765227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T09:28:34.774979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-11T09:28:45.573818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7000000476837158, 'score_stable': 1.7000000476837158, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}], 'osds': [{'osd': 0, 'uuid': '44ce15f5-731e-4d58-9dd7-bd8f186e5783', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 90, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6800', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6801', 'nonce': 4208334809}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6802', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6803', 'nonce': 4208334809}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6806', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6807', 'nonce': 4208334809}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6804', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6805', 'nonce': 4208334809}]}, 'public_addr': '172.21.15.17:6801/4208334809', 'cluster_addr': '172.21.15.17:6803/4208334809', 'heartbeat_back_addr': '172.21.15.17:6807/4208334809', 'heartbeat_front_addr': '172.21.15.17:6805/4208334809', 'state': ['exists']}, {'osd': 1, 'uuid': 'dba82e38-a74b-4d42-bad4-9b1e013d4da3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4687957763671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6816', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6817', 'nonce': 361445038}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6818', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6819', 'nonce': 361445038}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6822', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6823', 'nonce': 361445038}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6820', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6821', 'nonce': 361445038}]}, 'public_addr': '172.21.15.17:6817/361445038', 'cluster_addr': '172.21.15.17:6819/361445038', 'heartbeat_back_addr': '172.21.15.17:6823/361445038', 'heartbeat_front_addr': '172.21.15.17:6821/361445038', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64a33918-9bb2-4d59-a66e-73da15d78cbd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6808', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6809', 'nonce': 211521567}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6810', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6811', 'nonce': 211521567}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6814', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6815', 'nonce': 211521567}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6812', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6813', 'nonce': 211521567}]}, 'public_addr': '172.21.15.17:6809/211521567', 'cluster_addr': '172.21.15.17:6811/211521567', 'heartbeat_back_addr': '172.21.15.17:6815/211521567', 'heartbeat_front_addr': '172.21.15.17:6813/211521567', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a4774bc9-54ea-4272-b6b4-a6dd00c4c930', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 2070567113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 2070567113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 2070567113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 2070567113}]}, 'public_addr': '172.21.15.96:6801/2070567113', 'cluster_addr': '172.21.15.96:6803/2070567113', 'heartbeat_back_addr': '172.21.15.96:6807/2070567113', 'heartbeat_front_addr': '172.21.15.96:6805/2070567113', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a3984c6d-7250-4e85-a032-1dc0315642d8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 1795828331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 1795828331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 1795828331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 1795828331}]}, 'public_addr': '172.21.15.96:6817/1795828331', 'cluster_addr': '172.21.15.96:6819/1795828331', 'heartbeat_back_addr': '172.21.15.96:6823/1795828331', 'heartbeat_front_addr': '172.21.15.96:6821/1795828331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0a40bbb7-9d37-4a1b-b525-86608a3eaaa2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 90, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 3808455039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 3808455039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 3808455039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 3808455039}]}, 'public_addr': '172.21.15.96:6809/3808455039', 'cluster_addr': '172.21.15.96:6811/3808455039', 'heartbeat_back_addr': '172.21.15.96:6815/3808455039', 'heartbeat_front_addr': '172.21.15.96:6813/3808455039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.320025+0000', 'dead_epoch': 90}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.922939+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.876780+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.854159+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.081538+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.168719+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.18', 'mappings': [{'from': 0, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 39, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 39, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T09:30:38.490 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-11T09:30:38.504 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:38.505+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:38.513 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:38.513 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:38.514 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:38.514 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:38.522 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:38.605 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:38.606+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:38.670 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:38.670 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:38.670 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:38.670 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:38.670 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:38.679 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:38.706 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:38.707+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:38.787 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:38.787 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:38.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:38.787 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:38.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.876124309999994, 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:38.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:38.791 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.792 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.875893738000002, 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.793 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:38.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.795 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.796 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.875796534000003, 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.797 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.798 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:38.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.800 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.801 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.875561770000004, 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.802 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:38.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.804 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:38.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.875365754000001, 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.807 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.808 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:38.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.810 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:38.811 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.875097316999998, 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.812 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:38.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.815 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.874995566999999, 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.816 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.817 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:38.819 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.874601909999996, 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.821 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:38.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.824 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.825 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.874475665000006, 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.826 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.827 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:38.829 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.874266282999997, 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.830 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:38.831 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.832 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:38.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.874155333999994, 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.835 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:38.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.838 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.839 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.873899395999999, 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.840 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:38.842 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.843 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.873746866999994, 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.844 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:38.845 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:38.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.848 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.873554502999994, 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.849 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.850 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:38.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.852 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.853 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.873224085000004, 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.854 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:38.856 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.857 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.858 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.872947754999998, 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.859 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:38.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:38.861 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.872893834999999, 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:38.863 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:38.865 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.866 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:38.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.872693265999999, 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.868 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:38.870 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.871 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:38.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 87.872337727000001, 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.875 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:38.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:38.877 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 56.093072194000001, 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:38.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:38.879 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.880 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:38.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:38.882 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.883 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.884 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:38.885 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:38.885 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:38.885 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:38.886 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:38.887+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:38.887 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:38.987 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:38.988+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:39.036 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:39.039 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:39.039 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:39.039 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:39.039 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:39.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:39.054 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:39.054 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:39.071 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:39.072 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:39.088 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:39.088+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:39.089 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:39.089 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:39.107 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:39.108 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:39.125 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:39.125 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:39.128 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:39.128 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:39.158 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.158 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.168 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:39.189 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:39.189+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:39.290 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:39.291+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:39.315 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.315 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:39.391 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:39.391+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:39.426 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.427 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.435 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:39.492 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:39.493+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:39.578 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.579 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:39.630 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:39.631+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:39.689 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.690 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.694 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:39.695+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:39.698 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:39.794 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:39.795+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:39.841 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.842 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.850 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:39.934 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:39.935+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:39.993 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:39.993 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:39.996 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:39.996+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:40.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:40.028 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:30:40.096 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:40.097+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:40.154 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.154 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:40.196 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:40.197+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:40.297 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:40.298+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:40.306 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.307 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.315 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:40.398 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:40.399+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:40.458 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.458 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:40.499 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:40.500+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:40.569 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.569 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:40.600 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:40.600+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:40.690 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.690 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.700 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:40.701 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:40.701+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:40.803 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:40.803+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:40.807 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.807 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.816 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:40.903 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:40.904+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:40.923 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:40.923 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:40.932 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:41.004 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:41.006+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:41.022 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:30:41.046 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:41.047 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:41.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:41.105 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:41.105+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:41.129 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:41.206 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:41.206+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:41.206 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:41.206 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:41.216 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:41.247 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:41.265 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:41.265 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:41.282 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:41.283 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:41.299 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:41.300 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:41.307 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:41.307+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:41.316 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:41.316 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:41.327 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:41.327 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:41.327 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136649101000003, 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.328 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:41.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:41.331 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.332 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136628381999998, 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.333 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:41.334 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.335 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:41.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:41.337 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136607863999998, 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:41.338 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.339 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.340 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:41.341 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136585207000003, 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.342 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.343 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.344 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:41.345 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.346 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136559484999999, 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.347 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:41.349 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.350 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.351 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136539399, 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136505872999997, 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136482903000001, 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136462512999998, 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:41.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.367 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:41.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.370 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136442488999997, 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:41.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:41.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136422877999998, 2024-09-11T09:30:41.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:41.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:41.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136397844000001, 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.380 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:41.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 32.136373927000001, 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.385 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.389 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.473004861, 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.390 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:41.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.472982083000002, 2024-09-11T09:30:41.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:41.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.400 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.472954472000001, 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.401 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:41.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.472933566999998, 2024-09-11T09:30:41.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:41.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.472913220999999, 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:41.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.472729399999999, 2024-09-11T09:30:41.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:41.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:41.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:41.422 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.472699644999999, 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:41.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:41.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:41.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.427 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:41.428 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:41.428 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:41.429 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:41.429 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:41.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:41.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:41.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:41.431 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:41.430+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:41.529 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:41.529+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:41.533 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:41.533 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:41.533 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:41.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:41.630 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:41.630+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:41.684 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:41.684 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:41.684 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:41.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:41.734 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:41.733+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:41.832 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:41.832+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:41.837 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.837 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 131.400067318, 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.838 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.839 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:41.840 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.841 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.842 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:41.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.844 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.784161591, 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:41.845 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.846 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:41.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.848 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.849 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:41.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.783963890000003, 2024-09-11T09:30:41.851 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:41.852 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.853 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:41.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:41.857 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.783705464000001, 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.858 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:41.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:41.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:41.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.783675580000001, 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:41.864 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:41.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.783142427999998, 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:41.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.874 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.875 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.782264978999997, 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.782226012999999, 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.881 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:41.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.782182536999997, 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:41.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:41.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.890 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.78205792, 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:41.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:41.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.781492450000002, 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:41.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.781461813999996, 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:41.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.779325382000003, 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:41.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.909 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.777804893999999, 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.910 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:41.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.914 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.773731065, 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:41.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.773617270999999, 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.920 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:41.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:41.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.773532588999998, 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:41.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:41.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:41.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.932 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.773486597999998, 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:41.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.937 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.773115300000001, 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:41.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 54.772973350999997, 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:41.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:41.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:41.947 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:41.949 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:41.949 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:41.949+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:42.049 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:42.050+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:42.103 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:42.103 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:42.103 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:42.112 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:42.151 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:42.151+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:42.220 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:42.220 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:42.221 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:42.221 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:42.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:42.272 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:42.272+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993320308999998, 2024-09-11T09:30:42.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:42.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993297089999999, 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:42.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:42.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993269929, 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.343 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:42.345 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.346 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.347 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993249403999997, 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.348 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.349 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.350 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.351 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993222971000002, 2024-09-11T09:30:42.352 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:42.353 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.354 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.355 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:42.356 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993194819000003, 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.358 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.359 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:42.360 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.361 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993164438999997, 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.363 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:42.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.365 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.366 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993141365, 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.367 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:42.368 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.369 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993112003999997, 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:42.371 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.373 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:42.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:42.375 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993090778000003, 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.377 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.378 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:42.379 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.380 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993070484999997, 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.381 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.383 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:42.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.385 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993044437999998, 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.386 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:42.387 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.388 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:42.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993022857, 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:42.390 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.392 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.393 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:42.394 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.993002054000002, 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.395 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.397 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.398 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.399 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.992982644000001, 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:42.402 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.403 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.404 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.992945677000002, 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.405 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:42.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.407 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.408 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.992913274999999, 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.410 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:42.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.412 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.413 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.992891108000002, 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.414 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.415 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:42.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.417 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.992866128000003, 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:42.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:42.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:42.423 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 32.992845748999997, 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:42.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.428 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:42.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:42.430 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:42.429+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:42.530 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:42.530+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:42.572 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:42.573 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:42.573 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:42.573 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:42.581 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:42.665 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:42.665+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:42.724 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:42.724 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:42.724 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:42.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:42.733 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:42.733+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:42.833 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:42.834+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:42.876 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.893207696000005, 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.893161652000003, 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:42.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:42.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.893149981999997, 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:42.887 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:42.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:42.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:42.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891861712999997, 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:42.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.896 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891826011999996, 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.897 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:42.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.898 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:42.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.900 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:42.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:42.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:42.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891803975000002, 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:42.904 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.906 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:42.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891768389999996, 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:42.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:42.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:42.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:42.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.915 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891708046999995, 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.916 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891659363000002, 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:42.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:42.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:42.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891602899000006, 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:42.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:42.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:42.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.932 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891583343999997, 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.933 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891535580999999, 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:42.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:42.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:42.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891421843000003, 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:42.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:42.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891403099000001, 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:42.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:42.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:42.956 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891364109999998, 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:42.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.891341002999994, 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:42.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:42.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:42.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.183264511999994, 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:42.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:42.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:42.974 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.820865181999999, 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:42.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.815614105000002, 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:42.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:42.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:42.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:42.987 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.815353735000002, 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:42.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:42.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:42.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:42.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:42.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:42.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:42.995 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:42.997 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:42.996+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:42.997 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:43.044 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:30:43.097 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:43.098+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:43.125 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:43.125 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:43.126 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:43.135 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:43.198 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:43.198+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:43.284 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:43.284 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:43.284 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:43.284 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:43.284 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:43.293 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:43.336 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:43.337+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:43.401 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:43.401+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:43.403 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:43.403 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:43.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:43.403 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:43.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.403 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.404 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:43.404 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.490604653000005, 2024-09-11T09:30:43.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:43.404 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.404 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.405 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.405 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.405 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.405 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.405 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:43.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:43.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:43.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:43.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.412 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.490374080999999, 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.490276876999999, 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.490042113000001, 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.430 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.489846096999997, 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:43.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:43.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.489577659999995, 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:43.435 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.489475909999996, 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:43.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:43.444 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.489082253000007, 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.445 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:43.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:43.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.488956008000002, 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.450 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:43.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.488746625999994, 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.455 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:43.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:43.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.488635677000005, 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.460 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:43.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.464 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.488379738999996, 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.465 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:43.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.469 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.488227210000005, 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:43.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.488034846000005, 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.474 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:43.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.477 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:43.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3986 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 219136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.487704428000001, 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038816359000000002, 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3986 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913849+0000", 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3200000000000005e-07 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913852+0000", 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6809999999999998e-06 2024-09-11T09:30:43.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913861+0000", 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2949999999999999e-06 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913865+0000", 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3550000000000001e-06 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952489+0000", 2024-09-11T09:30:43.482 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038624054999999997 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952569+0000", 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0422999999999999e-05 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952665+0000", 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.5917999999999999e-05 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.483 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.487428097999995, 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.484 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.486 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.487374177999996, 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:43.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.491 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.487173608999996, 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.493 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:43.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.496 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:43.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:4006 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 452608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 92.486818069999998, 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849000000000002, 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.498 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4006 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914735+0000", 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914736+0000", 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9299999999999998e-07 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914738+0000", 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7819999999999998e-06 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914746+0000", 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.5510000000000002e-06 2024-09-11T09:30:43.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914750+0000", 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.49e-06 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.501 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953422+0000", 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038672326 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953500+0000", 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8332999999999997e-05 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953584+0000", 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3924999999999994e-05 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.502 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 60.707552536999998, 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:43.503 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:43.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:43.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:43.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:43.509 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:43.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:43.510 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:30:43.510 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:43.510 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-11T09:30:43.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-11T09:30:43.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:43.513 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:43.512+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:43.612 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:43.613+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:43.660 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:43.679 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:43.679 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:43.696 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:43.696 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:43.706 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:43.707 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:43.713 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:43.712+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:43.716 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:43.716 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:43.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:43.731 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:43.731 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:43.748 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:43.748 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:43.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:43.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:43.814 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:43.814+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:43.826 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:43.826 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:43.835 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:43.915 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:43.916+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:43.943 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:43.943 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:43.952 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:44.017 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:44.016+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:44.035 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:30:44.036 INFO:teuthology.orchestra.run.smithi017.stdout:moved 4 / 82 (4.87805%) 2024-09-11T09:30:44.036 INFO:teuthology.orchestra.run.smithi017.stdout:avg 16.4 2024-09-11T09:30:44.036 INFO:teuthology.orchestra.run.smithi017.stdout:stddev 3.29848 -> 4.00999 (expected baseline 3.62215) 2024-09-11T09:30:44.036 INFO:teuthology.orchestra.run.smithi017.stdout:min osd.2 with 11 -> 12 pgs (0.670732 -> 0.731707 * mean) 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:max osd.3 with 16 -> 17 pgs (0.97561 -> 1.03659 * mean) 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:oload 110 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:max_change 1 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:max_change_osds 4 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:average_utilization 0.0007 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:overload_utilization 0.0007 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 weight 1.0000 -> 0.7348 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 weight 1.0000 -> 0.7545 2024-09-11T09:30:44.037 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 weight 1.0000 -> 0.8791 2024-09-11T09:30:44.056 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-11T09:30:44.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-11T09:30:44.071 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.071 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:44.124 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:44.122+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:44.218 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:44.218+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:44.225 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.225 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:44.319 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:44.319+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:44.337 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.337 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.346 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:moved 0 / 82 (0%) 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:avg 16.4 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:stddev 4.00999 -> 4.00999 (expected baseline 3.62215) 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:min osd.4 with 10 -> 10 pgs (0.609756 -> 0.609756 * mean) 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:max osd.1 with 17 -> 17 pgs (1.03659 -> 1.03659 * mean) 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:oload 1000 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:max_change 3 2024-09-11T09:30:44.347 INFO:teuthology.orchestra.run.smithi017.stdout:max_change_osds 4 2024-09-11T09:30:44.348 INFO:teuthology.orchestra.run.smithi017.stdout:average_utilization 0.0007 2024-09-11T09:30:44.348 INFO:teuthology.orchestra.run.smithi017.stdout:overload_utilization 0.0067 2024-09-11T09:30:44.348 INFO:teuthology.orchestra.run.smithi017.stderr:no change 2024-09-11T09:30:44.360 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-11T09:30:44.360 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-11T09:30:44.444 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:44.445+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:44.450 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.450 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:44.521 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:44.521+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:44.562 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.562 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:44.622 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:44.621+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:44.674 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.674 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.683 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:44.723 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:44.723+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:44.824 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:44.824+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:44.826 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.826 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:44.925 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:44.926+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:44.978 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:44.978 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:44.987 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:45.030 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:45.030+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:moved 7 / 82 (8.53659%) 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:avg 16.4 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:stddev 4.00999 -> 5.02394 (expected baseline 3.62215) 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:min osd.4 with 10 -> 7 pgs (0.609756 -> 0.426829 * mean) 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:max osd.1 with 17 -> 17 pgs (1.03659 -> 1.03659 * mean) 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:oload 110 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:max_change 3 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:max_change_osds 4 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:average_utilization 0.0007 2024-09-11T09:30:45.038 INFO:teuthology.orchestra.run.smithi017.stdout:overload_utilization 0.0007 2024-09-11T09:30:45.039 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 weight 0.7348 -> 0.5399 2024-09-11T09:30:45.039 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 weight 0.7545 -> 0.5675 2024-09-11T09:30:45.039 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 weight 0.8791 -> 0.7732 2024-09-11T09:30:45.053 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-11T09:30:45.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-11T09:30:45.093 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:45.093 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:45.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:45.137 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:45.138+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:45.227 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:45.227+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:45.273 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:45.273 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:45.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:45.328 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:45.328+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:45.389 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:45.390 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:45.398 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:45.442 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:45.443+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:45.506 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:45.506 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:45.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:45.530 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:45.530+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:45.631 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:45.631+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:45.662 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:45.662 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:45.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:45.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:45.755 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:45.755+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:45.819 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:45.819 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:45.828 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:45.863 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:45.863 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:45.863+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:45.880 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:45.880 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:45.897 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:45.897 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:45.915 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:45.915 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:45.933 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:45.933 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:45.933 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:45.933+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:45.941 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749435484000003, 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.942 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:45.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749414764999997, 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.946 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:45.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:45.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749394246999998, 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.951 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.952 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:45.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:45.955 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749371590000003, 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.956 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.957 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:45.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749345867999999, 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.961 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.962 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:45.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.963 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:45.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:45.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749325782, 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.967 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:45.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.971 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749292255999997, 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.972 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:45.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.976 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749269286000001, 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.978 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.981 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749248895999997, 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:45.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749228872000003, 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.986 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:45.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:45.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:45.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749209260999997, 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.991 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.992 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:45.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:45.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749184227000001, 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:45.996 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:45.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:45.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:46.000 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 36.749160310000001, 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.001 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:46.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4330 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1419264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085791243999999, 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026843278000000002, 2024-09-11T09:30:46.005 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4330 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854258+0000", 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854259+0000", 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9700000000000002e-07 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854261+0000", 2024-09-11T09:30:46.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2000000000000001e-06 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854267+0000", 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3369999999999996e-06 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854272+0000", 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1869999999999996e-06 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879248+0000", 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024975329000000001 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879250+0000", 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1969999999999999e-06 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881012+0000", 2024-09-11T09:30:46.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0017622919999999999 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881053+0000", 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.07e-05 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881101+0000", 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8639e-05 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.010 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085768465999998, 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.011 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:46.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085740854999997, 2024-09-11T09:30:46.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:46.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:46.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:46.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.021 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085719949999998, 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:46.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:46.027 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085699603999998, 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:46.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:46.032 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085515782999998, 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.033 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:46.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.038 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.085486027999998, 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:46.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:46.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.041 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:46.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:46.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:46.044 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:46.046 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:46.046 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:46.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:46.047 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:46.047 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:46.047 INFO:teuthology.orchestra.run.smithi017.stdout:moved 6 / 82 (7.31707%) 2024-09-11T09:30:46.047 INFO:teuthology.orchestra.run.smithi017.stdout:avg 16.4 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:stddev 5.02394 -> 5.1342 (expected baseline 3.62215) 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:min osd.4 with 7 -> 7 pgs (0.426829 -> 0.426829 * mean) 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:max osd.3 with 18 -> 18 pgs (1.09756 -> 1.09756 * mean) 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:oload 110 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:max_change 3 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:max_change_osds 4 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:average_utilization 0.0007 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:overload_utilization 0.0007 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 weight 0.5399 -> 0.3967 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 weight 0.5675 -> 0.4268 2024-09-11T09:30:46.048 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 weight 0.7732 -> 0.6800 2024-09-11T09:30:46.049 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:46.045+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:46.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:30:46.058 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-11T09:30:46.059 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-11T09:30:46.146 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:46.145+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:46.187 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:46.187 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:46.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.187 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPush(3.19 118/117 [PushOp(3:9cc2ebca:::rbd_data.110684ff6664.0000000000000035:15, version: 58'217, data_included: [0~247808], data_size: 247808, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(3:9cc2ebca:::rbd_data.110684ff6664.0000000000000035:15@58'217, size: 247808, copy_subset: [(0, 247808)], clone_subset: {}, snapset: 15=[]:{15: [15]}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 247808, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:30:46.187 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:46.187436+0000", 2024-09-11T09:30:46.187 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.00039813899999999999, 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00045129300000000002, 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187436+0000", 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187436+0000", 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187436+0000", 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2699999999999999e-07 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187672+0000", 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00023644599999999999 2024-09-11T09:30:46.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187691+0000", 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8308000000000002e-05 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187709+0000", 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.7756000000000001e-05 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187783+0000", 2024-09-11T09:30:46.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.4406999999999994e-05 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:46.187791+0000", 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3899999999999993e-06 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:30:46.191 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-11T09:30:46.192 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:46.198 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:46.247 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:46.247+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:46.347 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:46.347+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:46.353 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:46.353 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:46.353 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:46.353 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:46.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:46.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:46.447 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:46.447+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:46.459 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-11T09:30:46.459+0000 7f7a27d23640 -1 mgr.server reply reply (3) No such process no change 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:moved 1 / 82 (1.21951%) 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:avg 16.4 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:stddev 5.1342 -> 5.50999 (expected baseline 3.62215) 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:min osd.4 with 7 -> 6 pgs (0.426829 -> 0.365854 * mean) 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:max osd.3 with 18 -> 18 pgs (1.09756 -> 1.09756 * mean) 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:oload 110 2024-09-11T09:30:46.460 INFO:teuthology.orchestra.run.smithi017.stdout:max_change 0.05 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stdout:max_change_osds 4 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stdout:average_utilization 0.0007 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stdout:overload_utilization 0.0007 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4 weight 0.3967 -> 0.3467 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5 weight 0.4268 -> 0.3768 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1 weight 0.6800 -> 0.6300 2024-09-11T09:30:46.461 INFO:teuthology.orchestra.run.smithi017.stderr:no change 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:46.509 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 136.07180434700001, 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:46.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:46.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:46.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.455898619999999, 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:46.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:46.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:46.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:46.523 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.455700919000002, 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:46.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.455442493, 2024-09-11T09:30:46.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:46.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:46.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.455412609, 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:46.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:46.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.454879456999997, 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:46.540 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:46.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.454002008000003, 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:46.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.549 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.453963041999998, 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:46.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.453919566000003, 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.554 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:46.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:46.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:46.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.453794948999999, 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:46.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:46.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.453229479000001, 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:46.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:46.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:46.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.567 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.453198843000003, 2024-09-11T09:30:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.569 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:46.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:46.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.451062411000002, 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.573 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:46.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:46.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.577 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.449541922999998, 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:46.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:46.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.445468093999999, 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:46.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.445354299999998, 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:46.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:46.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.445269617999998, 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:46.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:46.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:46.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.445223626999997, 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.601 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:46.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.605 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.444852329, 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:46.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 59.444710379999997, 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:46.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:46.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.616 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:46.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:46.618 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:46.618+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:46.718 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:46.719+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:46.765 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:46.765 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:46.765 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:46.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:46.819 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:46.820+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:46.876 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:46.876 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:46.876 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:46.884 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:46.920 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:46.920+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:46.988 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.988 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648696289, 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.989 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:46.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648673070000001, 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:46.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:46.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648645909000003, 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:46.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:46.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:47.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648625383999999, 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:47.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648598950999997, 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:47.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:47.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.012 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648570798999998, 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:47.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648540419, 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:47.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.021 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648517345000002, 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.022 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:47.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:47.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648487983999999, 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.027 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:47.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:47.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:47.031 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648466757999998, 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:47.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648446464999999, 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.040 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648420418000001, 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:47.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648398837000002, 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:47.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:47.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.049 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648378033999997, 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648358623999997, 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:47.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:47.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648321656999997, 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:47.058 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:47.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648289255000002, 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:47.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:47.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648267087999997, 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:47.067 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:47.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:47.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648242107999998, 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:47.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.076 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 37.648221728999999, 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:47.077 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:47.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.081 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:47.083 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:30:47.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:47.084 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:47.082+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:47.183 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:47.182+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:47.245 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:47.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "MOSDPGPush(3.14 119/118 [PushOp(3:2bb4acb3:::rbd_id.image_client.0-clone5:head, version: 32'30, data_included: [0~15], data_size: 15, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:2bb4acb3:::rbd_id.image_client.0-clone5:head@32'30, size: 15, copy_subset: [(0, 15)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 15, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:47.245828+0000", 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000181025, 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000208638, 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245828+0000", 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245828+0000", 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245829+0000", 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245841+0000", 2024-09-11T09:30:47.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2094e-05 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245860+0000", 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9530000000000001e-05 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245885+0000", 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4428999999999999e-05 2024-09-11T09:30:47.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245955+0000", 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0778000000000006e-05 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:47.245963+0000", 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3989999999999998e-06 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.249 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:30:47.250 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 1 2024-09-11T09:30:47.250 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:47.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:47.295 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:47.295+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:47.385 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:47.385+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:47.410 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:47.410 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:47.420 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:47.505 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:47.505+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:47.568 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:47.568 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.584980873000006, 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:47.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.584934829000005, 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.584923158999999, 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:47.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:47.583 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583634889999999, 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:47.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583599188999997, 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.588 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:47.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583577152000004, 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:47.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583541566999997, 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:47.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583481223999996, 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:47.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:47.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583432540000004, 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:47.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:47.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583376075999993, 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:47.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583356520999999, 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:47.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:47.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583308758000001, 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:47.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:47.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583195020000005, 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:47.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:47.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583176276000003, 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:47.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583137287, 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:47.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:47.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:47.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.583114179999995, 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:47.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.875037688999996, 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:47.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:47.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.512638359, 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:47.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:47.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.507387282000003, 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:47.663 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.507126911999997, 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:47.664 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:47.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:47.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:47.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:47.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:47.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:47.670 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:47.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:47.672 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:47.672+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:47.772 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:47.772+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:47.800 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "MOSDPGPushReply(3.b 120/119 [PushReplyOp(3:d701c77d:::rbd_data.1106541f23ef.0000000000000001:head)])", 2024-09-11T09:30:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.801122+0000", 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.00018761200000000001, 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000209879, 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "reached pg", 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801122+0000", 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801122+0000", 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801122+0000", 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2200000000000004e-07 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801129+0000", 2024-09-11T09:30:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0010000000000004e-06 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801139+0000", 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0019e-05 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801144+0000", 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1250000000000001e-06 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.801188+0000", 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3170999999999997e-05 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 1 2024-09-11T09:30:47.804 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:47.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:47.873 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:47.873+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:47.967 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:47.968 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:47.968 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:47.968 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:47.968 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:47.974 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:47.975+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:47.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:48.048 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:48.103 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:48.104+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.215606432000001, 2024-09-11T09:30:48.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:48.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:30:48.131 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.215375859999995, 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.132 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.133 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:30:48.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.215278655999995, 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.137 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.138 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:30:48.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.141 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.215043891999997, 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:30:48.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:48.146 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.214847875999993, 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:30:48.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:30:48.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:48.151 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.214579439000005, 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:30:48.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.214477689000006, 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:30:48.156 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:48.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:30:48.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.214084032000002, 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:30:48.161 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.162 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:30:48.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:30:48.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:48.166 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.213957786999998, 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.167 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:30:48.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.213748405000004, 2024-09-11T09:30:48.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:30:48.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.213637456000001, 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.176 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:30:48.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.213381518000006, 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.181 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:30:48.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:30:48.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.213228989000001, 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.188 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:30:48.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.213036625000001, 2024-09-11T09:30:48.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:48.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:30:48.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:30:48.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.212429877000005, 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:30:48.195 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:48.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:30:48.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.212375957000006, 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:30:48.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:30:48.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.204 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 97.212175388000006, 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:30:48.205 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:48.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:30:48.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 65.432554315999994, 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:30:48.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:30:48.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25886 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 2.0479905519999999, 2024-09-11T09:30:48.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97909465799999995, 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25886 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078566+0000", 2024-09-11T09:30:48.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6339999999999999e-06 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078569+0000", 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4690000000000002e-06 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078577+0000", 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1160000000000001e-06 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078583+0000", 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078638+0000", 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4812000000000003e-05 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078645+0000", 2024-09-11T09:30:48.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.686e-06 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057335+0000", 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97868998600000001 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057478+0000", 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014334499999999999 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057659+0000", 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018098100000000001 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25901 3.19 3:9f8aebfb:::rbd_data.11064477c615.0000000000000009:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 1.050810051, 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62440548799999995, 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:48.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25901 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075746+0000", 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4099999999999998e-07 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075750+0000", 2024-09-11T09:30:48.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4599999999999996e-06 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075766+0000", 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5579000000000001e-05 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075773+0000", 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6839999999999996e-06 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075846+0000", 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2606000000000002e-05 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075864+0000", 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.766e-05 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.695967+0000", 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62010379199999999 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696129+0000", 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00016192800000000001 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696251+0000", 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121199 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:48.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698192+0000", 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0019412120000000001 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700042+0000", 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001850317 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700107+0000", 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4455999999999994e-05 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:48.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700150+0000", 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3853999999999997e-05 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:48.230 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:48.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:48.232 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:48.232 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:48.233 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:48.233 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:48.233 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:48.234 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:48.234 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:48.234 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:48.232+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:48.235 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:48.235 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:48.251 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:48.251 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:48.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:48.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:48.332 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:48.333+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:48.342 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:48.342 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:48.351 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:48.434 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:48.434+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:48.458 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:48.458 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:48.466 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:48.550 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:48.551+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:48.614 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:48.614 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:48.623 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:48.667 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:48.668+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:48.737 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:48.737+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:48.740 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:48.740 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:48.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:48.838 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:48.839+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:48.892 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:48.892 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:48.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:48.941 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:48.942+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:49.039 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:49.040+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:49.043 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.043 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.053 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:49.085 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:30:49.141 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:49.141+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:49.196 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.196 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.205 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:49.242 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:49.242+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:49.342 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:49.343+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:49.347 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.347 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:49.443 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:49.443+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:49.458 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.459 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:49.544 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:49.545+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:49.611 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.611 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.620 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:49.663 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:49.664+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:49.692 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:30:49.731 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.731 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.739 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:49.781 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:49.781+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:49.846 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:49.847+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:49.882 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:49.883 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:49.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:49.947 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:49.948+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:50.003 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:50.003 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:50.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:50.055 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:50.056+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:50.119 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:50.119 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:50.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:50.149 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:50.149+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:50.250 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:50.251+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:50.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:50.276 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:50.276 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:50.285 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:50.361 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:50.361 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:50.362+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:50.395 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:50.395 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:50.413 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:50.413 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:50.430 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:50.430 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:50.447 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:50.447 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:50.447 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:50.447 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:50.448 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:50.448 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:50.452 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:50.452+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:50.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:50.464 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:50.464 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:50.467 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:50.468 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:50.553 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:50.553+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:50.570 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:50.570 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:50.570 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:50.570 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:50.571 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:50.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:50.654 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:50.654+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:50.689 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.498036114000001, 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:50.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.498015395000003, 2024-09-11T09:30:50.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:50.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497994876999996, 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.699 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:50.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497972220000001, 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:50.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:50.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497946497999997, 2024-09-11T09:30:50.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:50.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497926411999998, 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.713 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:50.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.717 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497892886000002, 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.718 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:50.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497869915999999, 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:50.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:50.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497849526000003, 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.728 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.732 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497829502000002, 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:50.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:50.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497809891000003, 2024-09-11T09:30:50.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:50.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:50.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497784856999999, 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:50.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.746 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 41.497760939999999, 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:50.747 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:50.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:50.751 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 39.834369096000003, 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:50.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:50.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 39.834341485000003, 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:50.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:50.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 39.834320579999996, 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:50.763 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:50.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 39.834300233999997, 2024-09-11T09:30:50.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:50.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.773 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 39.834116412999997, 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:50.774 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.775 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.776 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:50.777 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:50.778 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 39.834086657999997, 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.779 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:50.780 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.781 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.782 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.783 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 3.610790159, 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:30:50.784 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.785 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:50.786 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:30:50.788 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.789 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:30:50.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:50.791 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:50.792 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:50.793+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:50.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:50.894 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:50.894+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:50.899 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:50.899 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:50.899 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:50.908 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:50.994 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:50.995+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:51.009 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.009 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:51.010 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:51.010 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:51.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:51.095 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:51.096+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:51.122 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.122 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 140.68431640599999, 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:51.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:51.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.068410678999996, 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:51.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:51.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:51.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.068212978000005, 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:51.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:51.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.067954552000003, 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:51.143 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:51.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:51.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:51.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.067924668000003, 2024-09-11T09:30:51.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:51.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.067391516000001, 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:51.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:51.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:51.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.066514067, 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:51.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:51.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:51.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.066475100999995, 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:51.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.066431625000007, 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.168 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:51.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:51.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:51.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.066307007999995, 2024-09-11T09:30:51.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:51.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.065741537999997, 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.177 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:51.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.181 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.065710902000006, 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:51.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:51.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.063574470000006, 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.186 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:51.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.190 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.062053981999995, 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:51.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:51.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:51.195 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.057980153000003, 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:51.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.057866359000002, 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.200 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:51.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.057781676999994, 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.057735686000001, 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.057364387999996, 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:51.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:51.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:51.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 64.057222439, 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:51.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.225 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.226 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.226 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:51.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:51.228 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:51.227+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:51.328 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:51.329+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:51.370 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:51.370 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:51.370 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:51.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:51.430 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:51.430+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:51.473 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:30:51.473 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:51.473 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T09:30:51.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T09:30:51.480 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:51.480 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:51.480 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:51.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:51.530 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:51.531+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:51.593 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:51.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253709631, 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:51.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253686412, 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:51.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:51.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253659251000002, 2024-09-11T09:30:51.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:51.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:51.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253638725999998, 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:51.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253612293000003, 2024-09-11T09:30:51.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:51.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253584140999997, 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:51.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.622 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253553760999999, 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:51.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253530687000001, 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:51.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253501325999999, 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:51.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:51.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253480099999997, 2024-09-11T09:30:51.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:51.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253459806999999, 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:51.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.25343376, 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:51.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:51.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253412179000001, 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:51.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:51.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:51.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253391376000003, 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:51.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253371966000003, 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:51.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253334999000003, 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:51.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.669 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253302597000001, 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.670 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:51.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253280429999997, 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:51.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:51.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253255449999997, 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:51.679 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:51.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:51.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.683 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.253235070999999, 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.684 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:51.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:51.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:51.688 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:51.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:51.690 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:51.690+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:51.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:30:51.785 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:51.785 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:51.785 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 124, 2024-09-11T09:30:51.785 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "23a8bf95-caba-4fea-a762-61040a1c1b77", 2024-09-11T09:30:51.785 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-11T09:28:17.894525+0000", 2024-09-11T09:30:51.785 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-11T09:30:51.695017+0000", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2024-09-11T09:30:12.854131+0000", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2024-09-11T09:29:50.757033+0000", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5806080, 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "nodeep-scrub", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "noscrub", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 5, 2024-09-11T09:30:51.786 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 3, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 6, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "luminous", 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "luminous", 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "squid", 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_crimson": false, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2024-09-11T09:30:51.787 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": ".mgr", 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:28.765227+0000", 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:51.788 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:51.789 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "15", 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:51.790 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:51.791 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:51.792 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_max": 32, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": {} 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 5.880000114440918, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 5.880000114440918, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.17000000178813934, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1, 2024-09-11T09:30:51.793 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:34.774979+0000", 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:51.794 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2024-09-11T09:30:51.795 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:51.796 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 2, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 19, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:51.797 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:51.798 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:51.799 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 3.0099999904632568, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 3.0099999904632568, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 2.5, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 2.5, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:51.800 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "pool_client.0", 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:45.573818+0000", 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:51.801 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 32, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 32, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 32, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 32, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 32, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:51.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "124", 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 45, 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 124, 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:51.803 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:51.804 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:51.805 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 2.0699999332427979, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 2.0699999332427979, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1.7200000286102295, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1.7200000286102295, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:51.806 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "44ce15f5-731e-4d58-9dd7-bd8f186e5783", 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 0, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:51.807 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 11, 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 90, 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.808 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.809 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.810 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.811 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/4208334809", 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/4208334809", 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/4208334809", 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/4208334809", 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: "exists" 2024-09-11T09:30:51.812 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "dba82e38-a74b-4d42-bad4-9b1e013d4da3", 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0.680023193359375, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 0.4687957763671875, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 120, 2024-09-11T09:30:51.813 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.814 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.815 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2024-09-11T09:30:51.816 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2024-09-11T09:30:51.817 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6817/361445038", 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6819/361445038", 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6823/361445038", 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6821/361445038", 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.818 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "64a33918-9bb2-4d59-a66e-73da15d78cbd", 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 121, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:51.819 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.820 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.821 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.823 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/211521567", 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/211521567", 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/211521567", 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/211521567", 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:51.824 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a4774bc9-54ea-4272-b6b4-a6dd00c4c930", 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 121, 2024-09-11T09:30:51.825 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6800", 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6801", 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.826 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6802", 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6803", 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.827 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6806", 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6807", 2024-09-11T09:30:51.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6804", 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6805", 2024-09-11T09:30:51.829 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6801/2070567113", 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6803/2070567113", 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6807/2070567113", 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6805/2070567113", 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a3984c6d-7250-4e85-a032-1dc0315642d8", 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0.3966522216796875, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 121, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:51.831 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6816", 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6817", 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6818", 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6819", 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.833 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6822", 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6823", 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.834 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6820", 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6821", 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6817/1795828331", 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6819/1795828331", 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6823/1795828331", 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6821/1795828331", 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:51.836 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "0a40bbb7-9d37-4a1b-b525-86608a3eaaa2", 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0.426788330078125, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 120, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6808", 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6809", 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6810", 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6811", 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:51.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6814", 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6815", 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.840 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6812", 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6813", 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.841 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6809/3808455039", 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6811/3808455039", 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6815/3808455039", 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6813/3808455039", 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:51.842 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.320025+0000", 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 90 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:51.843 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.922939+0000", 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.876780+0000", 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:51.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.854159+0000", 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:51.845 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.081538+0000", 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:51.846 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.168719+0000", 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [ 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.18", 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "mappings": [ 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "from": 0, 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: "to": 1 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.847 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_primaries": [], 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [ 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "2.0", 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: 3, 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.848 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "2.6", 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: 3, 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.1", 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: 2, 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: 4 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.849 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.b", 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: 4, 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.e", 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: 5 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.850 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.10", 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: 2, 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: 5 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.17", 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: 2, 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: 5 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2024-09-11T09:30:51.851 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklist": {}, 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "range_blocklist": {}, 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [ 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.852 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "snaps": [ 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "begin": 31, 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "length": 15 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [ 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: "snaps": [ 2024-09-11T09:30:51.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "begin": 31, 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "length": 1 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {}, 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": { 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_enabled": false, 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_bucket_count": 0, 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "degraded_stretch_mode": 0, 2024-09-11T09:30:51.854 INFO:teuthology.orchestra.run.smithi017.stdout: "recovering_stretch_mode": 0, 2024-09-11T09:30:51.855 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_bucket": 0 2024-09-11T09:30:51.855 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:51.855 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:51.857 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:51.857 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:51.857 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:51.858 INFO:tasks.thrashosds.thrasher:j is {'epoch': 124, 'fsid': '23a8bf95-caba-4fea-a762-61040a1c1b77', 'created': '2024-09-11T09:28:17.894525+0000', 'modified': '2024-09-11T09:30:51.695017+0000', 'last_up_change': '2024-09-11T09:30:12.854131+0000', 'last_in_change': '2024-09-11T09:29:50.757033+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T09:28:28.765227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T09:28:34.774979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.009999990463257, 'score_stable': 3.009999990463257, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-11T09:28:45.573818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 124, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.069999933242798, 'score_stable': 2.069999933242798, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}], 'osds': [{'osd': 0, 'uuid': '44ce15f5-731e-4d58-9dd7-bd8f186e5783', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 90, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6800', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6801', 'nonce': 4208334809}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6802', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6803', 'nonce': 4208334809}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6806', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6807', 'nonce': 4208334809}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6804', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6805', 'nonce': 4208334809}]}, 'public_addr': '172.21.15.17:6801/4208334809', 'cluster_addr': '172.21.15.17:6803/4208334809', 'heartbeat_back_addr': '172.21.15.17:6807/4208334809', 'heartbeat_front_addr': '172.21.15.17:6805/4208334809', 'state': ['exists']}, {'osd': 1, 'uuid': 'dba82e38-a74b-4d42-bad4-9b1e013d4da3', 'up': 1, 'in': 1, 'weight': 0.680023193359375, 'primary_affinity': 0.4687957763671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6816', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6817', 'nonce': 361445038}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6818', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6819', 'nonce': 361445038}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6822', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6823', 'nonce': 361445038}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6820', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6821', 'nonce': 361445038}]}, 'public_addr': '172.21.15.17:6817/361445038', 'cluster_addr': '172.21.15.17:6819/361445038', 'heartbeat_back_addr': '172.21.15.17:6823/361445038', 'heartbeat_front_addr': '172.21.15.17:6821/361445038', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64a33918-9bb2-4d59-a66e-73da15d78cbd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6808', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6809', 'nonce': 211521567}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6810', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6811', 'nonce': 211521567}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6814', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6815', 'nonce': 211521567}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6812', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6813', 'nonce': 211521567}]}, 'public_addr': '172.21.15.17:6809/211521567', 'cluster_addr': '172.21.15.17:6811/211521567', 'heartbeat_back_addr': '172.21.15.17:6815/211521567', 'heartbeat_front_addr': '172.21.15.17:6813/211521567', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a4774bc9-54ea-4272-b6b4-a6dd00c4c930', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 2070567113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 2070567113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 2070567113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 2070567113}]}, 'public_addr': '172.21.15.96:6801/2070567113', 'cluster_addr': '172.21.15.96:6803/2070567113', 'heartbeat_back_addr': '172.21.15.96:6807/2070567113', 'heartbeat_front_addr': '172.21.15.96:6805/2070567113', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a3984c6d-7250-4e85-a032-1dc0315642d8', 'up': 1, 'in': 1, 'weight': 0.3966522216796875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 1795828331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 1795828331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 1795828331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 1795828331}]}, 'public_addr': '172.21.15.96:6817/1795828331', 'cluster_addr': '172.21.15.96:6819/1795828331', 'heartbeat_back_addr': '172.21.15.96:6823/1795828331', 'heartbeat_front_addr': '172.21.15.96:6821/1795828331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0a40bbb7-9d37-4a1b-b525-86608a3eaaa2', 'up': 1, 'in': 1, 'weight': 0.426788330078125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 3808455039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 3808455039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 3808455039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 3808455039}]}, 'public_addr': '172.21.15.96:6809/3808455039', 'cluster_addr': '172.21.15.96:6811/3808455039', 'heartbeat_back_addr': '172.21.15.96:6815/3808455039', 'heartbeat_front_addr': '172.21.15.96:6813/3808455039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.320025+0000', 'dead_epoch': 90}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.922939+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.876780+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.854159+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.081538+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.168719+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.18', 'mappings': [{'from': 0, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '2.0', 'osds': [3, 1]}, {'pgid': '2.6', 'osds': [3, 1]}, {'pgid': '3.1', 'osds': [2, 4]}, {'pgid': '3.b', 'osds': [4, 1]}, {'pgid': '3.e', 'osds': [1, 5]}, {'pgid': '3.10', 'osds': [2, 5]}, {'pgid': '3.17', 'osds': [2, 5]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 15}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 31, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T09:30:51.858 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-11T09:30:51.858 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:51.858 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:51.856+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:51.957 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:51.958+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:51.998 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:51.998 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:51.999 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:52.007 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:52.058 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:52.059+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:52.111 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.128054886, 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:52.203 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:52.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.128008842, 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:52.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12799717199999, 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:52.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:52.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:52.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12670890299999, 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12667320200001, 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.224 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:52.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:52.229 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126651165, 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:52.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:52.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:52.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12661558000001, 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.235 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:52.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:52.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:52.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12655523700001, 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:52.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126506553, 2024-09-11T09:30:52.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:52.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:52.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:52.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126450089, 2024-09-11T09:30:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:52.257 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:52.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:52.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:52.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:52.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:52.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12643053399999, 2024-09-11T09:30:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:52.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:52.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:52.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:52.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.277 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.279 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126382771, 2024-09-11T09:30:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:52.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:52.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:52.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:52.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:52.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.290 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:52.291 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126269033, 2024-09-11T09:30:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.292 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.296 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.297 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:52.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.299 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126250289, 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:52.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:52.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.304 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.305 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.12621129999999, 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.306 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:52.307 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.308 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.309 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.310 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 101.126188193, 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:52.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:52.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.314 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.315 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.418111702000004, 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.316 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.317 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.318 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:52.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.320 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.055712372000002, 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.324 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.325 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.326 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.327 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:52.328 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.329 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.050461295000005, 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:52.331 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.332 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:52.333 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.334 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.050200924999999, 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:52.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.336 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:52.337 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:52.338 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:52.339 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:52.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.341 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.342 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:52.342 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:52.342 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:52.343 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:52.344 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:52.344+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:52.443 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:52.444+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:52.453 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:52.453 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:52.453 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:52.461 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:52.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:52.544 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:52.544+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:52.569 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:52.570 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:52.570 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:52.570 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:52.570 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:52.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:52.611 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:52.628 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:52.629 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:52.645 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:52.646+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:52.646 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:52.646 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:52.663 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:52.663 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:52.680 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:52.681 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:52.688 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:52.688 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:52.698 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:52.698 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:52.699 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:52.701 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:52.702 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:52.705 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:30:52.747 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:52.747+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:52.816 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:52.816 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:52.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:52.869 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:52.870+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:52.936 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:52.937 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:52.942 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:52.949 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:52.949+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:53.050 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:53.051+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:53.090 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.090 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.099 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:53.151 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:53.152+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:53.202 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.202 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:53.252 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:53.253+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:53.313 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.313 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:53.365 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:53.366+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:53.424 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.425 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:53.454 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:53.455+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:53.555 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:53.556+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:53.577 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.577 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.586 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:53.656 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:53.657+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:53.687 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.687 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:53.780 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:53.781+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:53.847 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:53.847 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:53.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:53.898 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:53.898+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:53.958 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:53.959+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:54.000 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.000 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.009 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:54.092 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:54.093+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.152 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.160 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:54.160+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:54.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:54.260 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:54.261+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:54.303 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.303 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:54.361 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:54.361+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:54.432 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.432 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.442 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:54.461 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:54.462+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:54.549 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.550 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.558 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:54.603 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:54.604+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:54.663 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:54.663+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:54.666 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.666 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.675 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:54.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:54.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:30:54.764 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:54.764+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:54.834 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:54.837 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:54.838 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:54.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:54.852 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:54.853 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:54.865 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:54.865+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:54.870 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:54.889 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:54.890 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:54.906 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:54.907 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:54.938 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:54.939 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:54.942 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:54.942 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:54.965 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:54.966+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:54.968 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:54.968 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:54.968 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:54.968 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:54.968 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:54.980 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:55.066 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:55.067+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935910133999997, 2024-09-11T09:30:55.127 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:55.128 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.129 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:55.130 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:55.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.130 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.130 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:55.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935889414999998, 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.133 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.134 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.135 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:55.136 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.137 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935868896999999, 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.138 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.139 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.140 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.141 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.142 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935846240000004, 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.143 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:55.145 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.146 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:55.147 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935820518, 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:55.149 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.150 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935800432000001, 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:55.152 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:55.154 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.155 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.156 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935766905999998, 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.157 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:55.158 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.159 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:55.160 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.161 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935743936000001, 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.162 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.163 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:55.165 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935723545999998, 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.168 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.169 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:55.170 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.171 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935703521999997, 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.172 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:55.173 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:55.175 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935683910999998, 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.177 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:55.179 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.180 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.181 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935658877000002, 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.182 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.184 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 45.935634960000002, 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:55.188 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:55.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.272243115999999, 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:55.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.194 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:55.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:55.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:55.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.272215504999998, 2024-09-11T09:30:55.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:55.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.203 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.272194599999999, 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:55.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:55.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:55.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.272174253999999, 2024-09-11T09:30:55.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:55.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:55.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.213 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:55.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.271990432999999, 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.215 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:55.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 44.271960677999999, 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.221 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.222 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:55.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 8.0486641789999993, 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:30:55.226 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:30:55.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:30:55.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.231 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:30:55.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:55.233 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:55.235 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:55.235 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:55.235+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:55.336 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:55.336+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:55.340 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:55.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:55.340 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:55.340 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:55.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:55.436 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:55.437+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:55.452 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:55.452 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:55.452 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:55.452 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:55.452 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:55.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:55.537 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:55.538+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:55.605 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 145.16783017399999, 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.606 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:30:55.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:30:55.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:30:55.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.551924447000005, 2024-09-11T09:30:55.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:30:55.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:30:55.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.551726746, 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.620 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:30:55.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:30:55.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.551468319999998, 2024-09-11T09:30:55.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:30:55.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:55.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.551438435999998, 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:30:55.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.550905283999995, 2024-09-11T09:30:55.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:55.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.550027834999995, 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:30:55.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:30:55.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.549988869000003, 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:55.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:30:55.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:30:55.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.549945393000002, 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:30:55.652 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:55.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:30:55.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.549820776000004, 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:30:55.657 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:55.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:30:55.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.549255306000006, 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:30:55.662 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:30:55.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.549224670000001, 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:30:55.667 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:30:55.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:30:55.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.547088238000001, 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:30:55.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:30:55.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:30:55.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.545567750000004, 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:30:55.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.541493920999997, 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:30:55.682 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:55.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.541380126999996, 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:30:55.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:30:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.693 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.541295445000003, 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:30:55.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:30:55.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.541249453999995, 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:30:55.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.704 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.540878156000005, 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:30:55.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:55.709 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 68.540736206999995, 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:55.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:55.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:30:55.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:55.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:55.714 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:55.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:55.716 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:55.716+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:55.717 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:30:55.816 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:55.816+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:55.867 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:55.867 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:55.867 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:55.867 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:55.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:55.917 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:55.918+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:56.018 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:56.018+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:56.019 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:56.019 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:56.019 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:56.019 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:56.019 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:56.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:56.119 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:56.120+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:56.172 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.832039705, 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:56.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:30:56.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.832016486000001, 2024-09-11T09:30:56.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:56.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:30:56.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831989325000002, 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:56.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.187 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831968799999999, 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.188 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:56.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:30:56.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.192 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831942367000003, 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:30:56.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:30:56.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831914214999998, 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:30:56.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:30:56.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:30:56.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:56.203 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831883834999999, 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:30:56.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831860761000002, 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:56.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:30:56.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831831399999999, 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:30:56.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831810173999997, 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.222 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831789880999999, 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831763834, 2024-09-11T09:30:56.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831742253000002, 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:30:56.232 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:56.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:30:56.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831721450000003, 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:30:56.237 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:30:56.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831702040000003, 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:30:56.246 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831665073000003, 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.247 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:30:56.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.251 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831632671000001, 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:30:56.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:30:56.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831610503999997, 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:30:56.256 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:56.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:30:56.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831585523999998, 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:30:56.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.265 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 46.831565144999999, 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.270 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:56.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:56.273 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:56.272+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:56.373 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:56.373+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:56.376 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:56.376 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:56.376 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:56.376 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:56.385 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:56.474 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:56.475+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:56.526 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:56.526 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:56.527 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:56.527 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:56.535 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:56.578 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:56.579+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:56.639 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.656063418, 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:30:56.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:56.645 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.656017374, 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:30:56.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:30:56.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:56.650 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65600570399999, 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:56.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65471743499999, 2024-09-11T09:30:56.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:56.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:30:56.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:30:56.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65468173399999, 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:30:56.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:30:56.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.665 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654659697, 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:30:56.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:30:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.671 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65462411199999, 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.672 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:30:56.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:30:56.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.676 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65456376900001, 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:30:56.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:30:56.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654515085, 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:30:56.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654458621, 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.686 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:30:56.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:30:56.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65443906599999, 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.691 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:30:56.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.695 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654391303, 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:30:56.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:30:56.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:56.700 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654277565, 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:30:56.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654258821, 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.705 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:30:56.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.709 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.654219832, 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:30:56.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:30:56.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.65419672500001, 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:30:56.714 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.716 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:30:56.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.946120234000006, 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.719 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:30:56.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:30:56.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:56.724 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.583720904000003, 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:30:56.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:30:56.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:56.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.730 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.578469827000006, 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:30:56.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:30:56.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:30:56.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:56.735 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.578209457, 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:30:56.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.739 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:30:56.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:30:56.741 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:30:56.742 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:56.743 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:56.743+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:56.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:56.844 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:56.844+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:56.859 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:30:56.859 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:30:56.859 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-11T09:30:56.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-11T09:30:56.865 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:56.865 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:56.865 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:56.865 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:56.874 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:56.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:30:56.961 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:56.962+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:57.030 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:57.031 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:57.031 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:57.040 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:57.070 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:57.070 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:57.071+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:57.088 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:57.106 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:57.106 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:57.123 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:57.123 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:57.140 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:57.140 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:57.146 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:57.146+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:57.150 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:57.150 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:57.157 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:57.157 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:57.159 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:57.160 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:57.161 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 129, 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "23a8bf95-caba-4fea-a762-61040a1c1b77", 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-11T09:28:17.894525+0000", 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-11T09:30:56.718188+0000", 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "last_up_change": "2024-09-11T09:30:12.854131+0000", 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "last_in_change": "2024-09-11T09:29:50.757033+0000", 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_num": 5799936, 2024-09-11T09:30:57.165 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_set": [ 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "pglog_hardlimit", 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "purged_snapdirs", 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_deletes", 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "sortbitwise" 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_version": 5, 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "full_ratio": 0.94999998807907104, 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_snapshot": "", 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_max": 3, 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "max_osd": 6, 2024-09-11T09:30:57.166 INFO:teuthology.orchestra.run.smithi017.stdout: "require_min_compat_client": "luminous", 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min_compat_client": "luminous", 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "require_osd_release": "squid", 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "allow_crimson": false, 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "pools": [ 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 1, 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": ".mgr", 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:28.765227+0000", 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 1, 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool", 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:57.167 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 1, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 1, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 1, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 1, 2024-09-11T09:30:57.168 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 1, 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "15", 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:57.169 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 0, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 0, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:57.170 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:57.171 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "options": { 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_max": 32, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_min": 1 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.172 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": {} 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 5.880000114440918, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 5.880000114440918, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.17000000178813934, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:57.173 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 2, 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "rbd", 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:34.774979+0000", 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:57.174 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:57.175 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 8, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 8, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 8, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 8, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 8, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:57.176 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "19", 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 2, 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 19, 2024-09-11T09:30:57.177 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:57.178 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:57.179 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 3.0099999904632568, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 3.0099999904632568, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:57.180 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 2.5, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 2.5, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_name": "pool_client.0", 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "create_time": "2024-09-11T09:28:45.573818+0000", 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flags": 8193, 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-11T09:30:57.181 INFO:teuthology.orchestra.run.smithi017.stdout: "type": 1, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 2, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "min_size": 1, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_rule": 0, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_count": 0, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_target": 0, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_barrier": 0, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "is_stretch_pool": false, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "object_hash": 2, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_autoscale_mode": "off", 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num": 32, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num": 32, 2024-09-11T09:30:57.182 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_placement_num_target": 32, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_target": 32, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_num_pending": 32, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "last_pg_merge_meta": { 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "source_pgid": "0.0", 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "ready_epoch": 0, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_started": 0, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "last_epoch_clean": 0, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "source_version": "0'0", 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "target_version": "0'0" 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "last_change": "129", 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend": "0", 2024-09-11T09:30:57.183 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "last_force_op_resend_preluminous": "0", 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "auid": 0, 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_mode": "selfmanaged", 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_seq": 45, 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "snap_epoch": 129, 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "pool_snaps": [], 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps": "[]", 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_bytes": 0, 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "quota_max_objects": 0, 2024-09-11T09:30:57.184 INFO:teuthology.orchestra.run.smithi017.stdout: "tiers": [], 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "tier_of": -1, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "read_tier": -1, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "write_tier": -1, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_mode": "none", 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_bytes": 0, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "target_max_objects": 0, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_flush_age": 0, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "cache_min_evict_age": 0, 2024-09-11T09:30:57.185 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profile": "", 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_params": { 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "none" 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_period": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_count": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "use_gmt_hitset": true, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "min_read_recency_for_promote": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "min_write_recency_for_promote": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_grade_decay_rate": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "hit_set_search_last_n": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "grade_table": [], 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "stripe_width": 0, 2024-09-11T09:30:57.186 INFO:teuthology.orchestra.run.smithi017.stdout: "expected_num_objects": 0, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "fast_read": false, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "options": {}, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "application_metadata": { 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "rbd": {} 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "read_balance": { 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "score_type": "Fair distribution", 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "score_acting": 2.0699999332427979, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "score_stable": 2.0699999332427979, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "optimal_score": 0.82999998331069946, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_acting": 1.7200000286102295, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "raw_score_stable": 1.7200000286102295, 2024-09-11T09:30:57.187 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity_weighted": 0.91146600246429443, 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity": 0.9100000262260437, 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "average_primary_affinity_weighted": 0.9100000262260437 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "44ce15f5-731e-4d58-9dd7-bd8f186e5783", 2024-09-11T09:30:57.188 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 0, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 11, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 74, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 90, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6800", 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6801", 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:57.190 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6802", 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6803", 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.191 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6806", 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6807", 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6804", 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6805", 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 4208334809 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.193 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6801/4208334809", 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6803/4208334809", 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6807/4208334809", 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6805/4208334809", 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "exists" 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "dba82e38-a74b-4d42-bad4-9b1e013d4da3", 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:57.194 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0.680023193359375, 2024-09-11T09:30:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 0.4687957763671875, 2024-09-11T09:30:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:57.195 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 120, 2024-09-11T09:30:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:57.196 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6816", 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6817", 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6818", 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6819", 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6822", 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6823", 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6820", 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6821", 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 361445038 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6817/361445038", 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6819/361445038", 2024-09-11T09:30:57.201 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6823/361445038", 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6821/361445038", 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "64a33918-9bb2-4d59-a66e-73da15d78cbd", 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 121, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6808", 2024-09-11T09:30:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6809", 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6810", 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6811", 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6814", 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6815", 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6812", 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6813", 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 211521567 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6809/211521567", 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.17:6811/211521567", 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.17:6815/211521567", 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.17:6813/211521567", 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a4774bc9-54ea-4272-b6b4-a6dd00c4c930", 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 1, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 121, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6800", 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6801", 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6802", 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6803", 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6806", 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6807", 2024-09-11T09:30:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6804", 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6805", 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 2070567113 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6801/2070567113", 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6803/2070567113", 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6807/2070567113", 2024-09-11T09:30:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6805/2070567113", 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "a3984c6d-7250-4e85-a032-1dc0315642d8", 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0.3966522216796875, 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 121, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6816", 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6817", 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6818", 2024-09-11T09:30:57.217 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6819", 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6822", 2024-09-11T09:30:57.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6823", 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6820", 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6821", 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1795828331 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6817/1795828331", 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6819/1795828331", 2024-09-11T09:30:57.220 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6823/1795828331", 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6821/1795828331", 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "uuid": "0a40bbb7-9d37-4a1b-b525-86608a3eaaa2", 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "up": 1, 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "in": 1, 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0.426788330078125, 2024-09-11T09:30:57.221 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_affinity": 1, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_begin": 0, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "last_clean_end": 0, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "up_from": 12, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "up_thru": 120, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "down_at": 0, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "lost_at": 0, 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6808", 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6809", 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addrs": { 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6810", 2024-09-11T09:30:57.223 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6811", 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addrs": { 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6814", 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6815", 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addrs": { 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-11T09:30:57.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6812", 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.96:6813", 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3808455039 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.96:6809/3808455039", 2024-09-11T09:30:57.226 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_addr": "172.21.15.96:6811/3808455039", 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_back_addr": "172.21.15.96:6815/3808455039", 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "heartbeat_front_addr": "172.21.15.96:6813/3808455039", 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "state": [ 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "exists", 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "up" 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_xinfo": [ 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 0, 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:57.227 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.320025+0000", 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 90 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 1, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:57.228 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.922939+0000", 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 2, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.876780+0000", 2024-09-11T09:30:57.229 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 3, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:25.854159+0000", 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.230 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 4, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.081538+0000", 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": 5, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "down_stamp": "0.000000", 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_probability": 0, 2024-09-11T09:30:57.231 INFO:teuthology.orchestra.run.smithi017.stdout: "laggy_interval": 0, 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "features": 4540701547738038271, 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "old_weight": 0, 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "last_purged_snaps_scrub": "2024-09-11T09:28:26.168719+0000", 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "dead_epoch": 0 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap": [], 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_items": [ 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.18", 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: "mappings": [ 2024-09-11T09:30:57.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "from": 0, 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "to": 1 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_upmap_primaries": [], 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "pg_temp": [ 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "2.0", 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: 3, 2024-09-11T09:30:57.233 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "2.6", 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: 3, 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.1", 2024-09-11T09:30:57.234 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: 2, 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: 4 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.b", 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: 4, 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.e", 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: 5 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.10", 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: 2, 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: 5 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "pgid": "3.17", 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "osds": [ 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: 2, 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: 5 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "primary_temp": [], 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklist": {}, 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "range_blocklist": {}, 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "erasure_code_profiles": { 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "default": { 2024-09-11T09:30:57.237 INFO:teuthology.orchestra.run.smithi017.stdout: "crush-failure-domain": "osd", 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "k": "2", 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "m": "1", 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "plugin": "jerasure", 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "technique": "reed_sol_van" 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_snaps_queue": [ 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "snaps": [ 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "begin": 26, 2024-09-11T09:30:57.238 INFO:teuthology.orchestra.run.smithi017.stdout: "length": 20 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "new_removed_snaps": [ 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "pool": 3, 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: "snaps": [ 2024-09-11T09:30:57.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "begin": 26, 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "length": 1 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "new_purged_snaps": [], 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_node_flags": {}, 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "device_class_flags": {}, 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": { 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_enabled": false, 2024-09-11T09:30:57.240 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_bucket_count": 0, 2024-09-11T09:30:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "degraded_stretch_mode": 0, 2024-09-11T09:30:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "recovering_stretch_mode": 0, 2024-09-11T09:30:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode_bucket": 0 2024-09-11T09:30:57.241 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:57.241 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:57.245 INFO:tasks.thrashosds.thrasher:j is {'epoch': 129, 'fsid': '23a8bf95-caba-4fea-a762-61040a1c1b77', 'created': '2024-09-11T09:28:17.894525+0000', 'modified': '2024-09-11T09:30:56.718188+0000', 'last_up_change': '2024-09-11T09:30:12.854131+0000', 'last_in_change': '2024-09-11T09:29:50.757033+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T09:28:28.765227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-11T09:28:34.774979+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.009999990463257, 'score_stable': 3.009999990463257, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-11T09:28:45.573818+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.069999933242798, 'score_stable': 2.069999933242798, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.7200000286102295, 'raw_score_stable': 1.7200000286102295, 'primary_affinity_weighted': 0.9114660024642944, 'average_primary_affinity': 0.9100000262260437, 'average_primary_affinity_weighted': 0.9100000262260437}}], 'osds': [{'osd': 0, 'uuid': '44ce15f5-731e-4d58-9dd7-bd8f186e5783', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 74, 'down_at': 90, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6800', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6801', 'nonce': 4208334809}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6802', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6803', 'nonce': 4208334809}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6806', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6807', 'nonce': 4208334809}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6804', 'nonce': 4208334809}, {'type': 'v1', 'addr': '172.21.15.17:6805', 'nonce': 4208334809}]}, 'public_addr': '172.21.15.17:6801/4208334809', 'cluster_addr': '172.21.15.17:6803/4208334809', 'heartbeat_back_addr': '172.21.15.17:6807/4208334809', 'heartbeat_front_addr': '172.21.15.17:6805/4208334809', 'state': ['exists']}, {'osd': 1, 'uuid': 'dba82e38-a74b-4d42-bad4-9b1e013d4da3', 'up': 1, 'in': 1, 'weight': 0.680023193359375, 'primary_affinity': 0.4687957763671875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6816', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6817', 'nonce': 361445038}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6818', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6819', 'nonce': 361445038}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6822', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6823', 'nonce': 361445038}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6820', 'nonce': 361445038}, {'type': 'v1', 'addr': '172.21.15.17:6821', 'nonce': 361445038}]}, 'public_addr': '172.21.15.17:6817/361445038', 'cluster_addr': '172.21.15.17:6819/361445038', 'heartbeat_back_addr': '172.21.15.17:6823/361445038', 'heartbeat_front_addr': '172.21.15.17:6821/361445038', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '64a33918-9bb2-4d59-a66e-73da15d78cbd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6808', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6809', 'nonce': 211521567}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6810', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6811', 'nonce': 211521567}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6814', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6815', 'nonce': 211521567}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6812', 'nonce': 211521567}, {'type': 'v1', 'addr': '172.21.15.17:6813', 'nonce': 211521567}]}, 'public_addr': '172.21.15.17:6809/211521567', 'cluster_addr': '172.21.15.17:6811/211521567', 'heartbeat_back_addr': '172.21.15.17:6815/211521567', 'heartbeat_front_addr': '172.21.15.17:6813/211521567', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a4774bc9-54ea-4272-b6b4-a6dd00c4c930', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6800', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6801', 'nonce': 2070567113}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6802', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6803', 'nonce': 2070567113}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6806', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6807', 'nonce': 2070567113}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6804', 'nonce': 2070567113}, {'type': 'v1', 'addr': '172.21.15.96:6805', 'nonce': 2070567113}]}, 'public_addr': '172.21.15.96:6801/2070567113', 'cluster_addr': '172.21.15.96:6803/2070567113', 'heartbeat_back_addr': '172.21.15.96:6807/2070567113', 'heartbeat_front_addr': '172.21.15.96:6805/2070567113', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a3984c6d-7250-4e85-a032-1dc0315642d8', 'up': 1, 'in': 1, 'weight': 0.3966522216796875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6816', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6817', 'nonce': 1795828331}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6818', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6819', 'nonce': 1795828331}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6822', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6823', 'nonce': 1795828331}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6820', 'nonce': 1795828331}, {'type': 'v1', 'addr': '172.21.15.96:6821', 'nonce': 1795828331}]}, 'public_addr': '172.21.15.96:6817/1795828331', 'cluster_addr': '172.21.15.96:6819/1795828331', 'heartbeat_back_addr': '172.21.15.96:6823/1795828331', 'heartbeat_front_addr': '172.21.15.96:6821/1795828331', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '0a40bbb7-9d37-4a1b-b525-86608a3eaaa2', 'up': 1, 'in': 1, 'weight': 0.426788330078125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 120, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6808', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6809', 'nonce': 3808455039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6810', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6811', 'nonce': 3808455039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6814', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6815', 'nonce': 3808455039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6812', 'nonce': 3808455039}, {'type': 'v1', 'addr': '172.21.15.96:6813', 'nonce': 3808455039}]}, 'public_addr': '172.21.15.96:6809/3808455039', 'cluster_addr': '172.21.15.96:6811/3808455039', 'heartbeat_back_addr': '172.21.15.96:6815/3808455039', 'heartbeat_front_addr': '172.21.15.96:6813/3808455039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.320025+0000', 'dead_epoch': 90}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.922939+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.876780+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:25.854159+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.081538+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-11T09:28:26.168719+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.18', 'mappings': [{'from': 0, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '2.0', 'osds': [3, 1]}, {'pgid': '2.6', 'osds': [3, 1]}, {'pgid': '3.1', 'osds': [2, 4]}, {'pgid': '3.b', 'osds': [4, 1]}, {'pgid': '3.e', 'osds': [1, 5]}, {'pgid': '3.10', 'osds': [2, 5]}, {'pgid': '3.17', 'osds': [2, 5]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 20}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-11T09:30:57.245 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-11T09:30:57.246 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:57.247+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:57.271 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:57.271 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:57.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:57.363 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:57.364+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:57.427 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:57.427 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:57.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:57.477 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:57.477+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:57.549 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:57.550+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:57.582 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:57.582 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:57.591 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:57.652 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:57.650+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:57.700 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:57.700 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:57.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:57.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:30:57.751 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:57.751+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:57.850 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:57.851+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:57.855 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:57.855 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:57.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:30:57.951 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:57.952+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:58.008 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.008 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:30:58.052 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:58.053+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:58.152 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:58.153+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:58.159 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.159 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.168 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:30:58.253 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:58.254+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:58.310 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.310 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.319 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:30:58.362 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:58.363+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:58.421 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.422 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.431 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:30:58.473 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:58.473+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:58.556 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:58.556+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:58.575 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.575 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.584 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:30:58.657 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:58.657+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:58.687 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.687 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:30:58.731 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:30:58.757 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:58.758+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:58.805 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.806 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.815 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:30:58.859 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:58.860+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:58.932 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:58.932 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:58.941 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:30:58.960 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:58.960+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:59.060 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:59.061+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:59.088 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:59.088 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:59.097 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:30:59.161 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:59.161+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:59.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:30:59.204 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:30:59.205 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:30:59.214 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:30:59.261 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:30:59.262+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:30:59.269 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:30:59.287 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:30:59.287 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:30:59.305 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:30:59.305 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:30:59.322 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:30:59.322 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:30:59.339 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:30:59.339 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:30:59.356 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:30:59.357 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:30:59.360 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:30:59.360 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:30:59.362 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:30:59.363+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:30:59.364 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:30:59.364 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:59.373 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:30:59.463 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:30:59.464+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:30:59.521 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:59.521 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:30:59.521 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:30:59.521 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:30:59.521 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:59.531 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:30:59.574 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:59.575+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:59.639 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448350470999998, 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.640 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.641 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:30:59.642 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.643 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:30:59.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448329751999999, 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.647 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.648 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:30:59.649 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:59.650 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448309234, 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.652 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:30:59.653 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:30:59.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448286576999998, 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.656 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:30:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448260855000001, 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:30:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448240769000002, 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:30:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:30:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448207242999999, 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:30:59.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.674 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448184273000003, 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:30:59.675 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:59.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:59.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448163882999999, 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.680 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:30:59.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:30:59.684 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448143858999998, 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.685 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:30:59.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:30:59.689 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.690 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.690 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448124247999999, 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.692 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:30:59.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.695 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:30:59.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448099214000003, 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.697 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:30:59.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.700 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:30:59.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 50.448075297000003, 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.702 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:30:59.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:30:59.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 48.784683453, 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.707 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:30:59.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:30:59.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 48.784655841999999, 2024-09-11T09:30:59.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:59.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:30:59.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.716 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:30:59.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 48.784634937, 2024-09-11T09:30:59.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:30:59.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:30:59.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:30:59.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 48.784614591, 2024-09-11T09:30:59.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:59.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:30:59.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 48.78443077, 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:59.731 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:30:59.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:30:59.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.736 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 48.784401015, 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:30:59.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:30:59.741 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 12.561104516, 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:30:59.742 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:30:59.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:30:59.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:30:59.748 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:30:59.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:30:59.749 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:30:59.750 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:30:59.751 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:59.750+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:59.851 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:30:59.851+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:30:59.867 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:59.867 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:59.867 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:30:59.867 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:59.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:30:59.951 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:30:59.952+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:30:59.979 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:30:59.980 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:30:59.980 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:30:59.980 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:30:59.980 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:30:59.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:00.053 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:00.053+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:00.093 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:00.093 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:00.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:00.093 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:00.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.093 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 149.655524144, 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.094 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:31:00.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:31:00.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:31:00.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.039618417, 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:31:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.109 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.039420715999995, 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.039162289999993, 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:31:00.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:00.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.123 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.039132405999993, 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.124 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.128 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.038599254000005, 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.129 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.133 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.037721805000004, 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:31:00.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.037682838999999, 2024-09-11T09:31:00.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:00.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:31:00.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:00.143 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.037639362999997, 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.147 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.037514745999999, 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:00.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:31:00.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:31:00.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.036949276000001, 2024-09-11T09:31:00.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:31:00.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:00.158 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.036918639999996, 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:31:00.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.161 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.162 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.034782207999996, 2024-09-11T09:31:00.163 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.164 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:31:00.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.166 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:31:00.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:00.168 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.033261719999999, 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:31:00.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.173 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.029187891000007, 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.175 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.029074097000006, 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:31:00.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:31:00.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.183 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.184 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.028989414999998, 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:31:00.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.190 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.028943424000005, 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:31:00.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.028572126, 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.200 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.028430177000004, 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:00.201 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:31:00.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.205 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:00.207 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:00.207 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:00.207+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:00.308 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:00.309+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:00.350 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:00.351 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:00.351 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:00.351 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:00.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:00.409 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:00.409+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:00.463 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:00.463 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:00.463 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:00.472 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:00.510 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:00.511+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:00.577 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236813386000001, 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.579 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:00.580 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.582 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.583 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236790167000002, 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:31:00.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.587 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236763005999997, 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:31:00.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:31:00.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236742481, 2024-09-11T09:31:00.593 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:31:00.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:31:00.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236716047999998, 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236687895999999, 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:31:00.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.607 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236657516000001, 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:00.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:31:00.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.612 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236634442000003, 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.613 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236605081, 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.618 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:31:00.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:00.622 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236583854999999, 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.623 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:31:00.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236563562000001, 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:31:00.629 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.630 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236537515000002, 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.636 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236515934000003, 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.637 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:31:00.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:31:00.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.641 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236495130999998, 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:31:00.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.644 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236475720999998, 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:31:00.646 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:31:00.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236438753999998, 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.651 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:31:00.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.655 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236406352000003, 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.656 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:00.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236384184999999, 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:31:00.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236359204999999, 2024-09-11T09:31:00.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:31:00.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.668 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:31:00.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 51.236338826000001, 2024-09-11T09:31:00.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:31:00.674 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.675 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.675 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:00.676 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:00.677 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:00.677+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:00.754 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:31:00.777 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:00.777+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:00.787 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:00.787 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:27598 3.1c 3:3dd43343:::rbd_object_map.110660e40da5:head [call lock.lock in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:31:00.787534+0000", 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0011253910000000001, 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0011386269999999999, 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 27598 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787534+0000", 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.789 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787534+0000", 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787535+0000", 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0299999999999998e-07 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.790 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787539+0000", 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1989999999999999e-06 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787547+0000", 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9470000000000003e-06 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787552+0000", 2024-09-11T09:31:00.791 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8350000000000003e-06 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787620+0000", 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8305000000000004e-05 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787674+0000", 2024-09-11T09:31:00.792 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3708999999999997e-05 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.787787+0000", 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00011273 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788463+0000", 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067641700000000001 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.793 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788618+0000", 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00015510599999999999 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788649+0000", 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0679999999999998e-05 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.794 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788673+0000", 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3995999999999999e-05 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_repop_reply(client.4358.0:27598 3.1c e133/21 ondisk, result = 0)", 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:31:00.788554+0000", 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.000105602, 2024-09-11T09:31:00.795 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00019521499999999999, 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788554+0000", 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:00.796 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788554+0000", 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788554+0000", 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5199999999999997e-07 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788558+0000", 2024-09-11T09:31:00.797 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5090000000000002e-06 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788566+0000", 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9300000000000003e-06 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788570+0000", 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4070000000000003e-06 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.798 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788609+0000", 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8525000000000001e-05 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:00.788614+0000", 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1150000000000002e-06 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-11T09:31:00.800 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 2 2024-09-11T09:31:00.800 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:00.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:00.877 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:00.878+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:00.945 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:00.946 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:00.946 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:00.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:00.979 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:00.979+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:01.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.07458634, 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:31:01.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.074540296, 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:31:01.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:31:01.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.074528626, 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.069 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:31:01.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:01.074 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.073240357, 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:31:01.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:31:01.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.073204656, 2024-09-11T09:31:01.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:01.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:31:01.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:31:01.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:31:01.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.07318261899999, 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:31:01.085 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:31:01.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:31:01.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.073147034, 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:01.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:31:01.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:01.095 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.073086691, 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:31:01.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.07303800699999, 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:31:01.100 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:31:01.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.072981543, 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:31:01.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.072961988, 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:31:01.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:31:01.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.07291422500001, 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.115 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:31:01.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:31:01.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:01.119 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.07280048699999, 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:01.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:31:01.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.07278174299999, 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:31:01.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:31:01.128 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.072742754, 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.129 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:01.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.131 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.133 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 110.072719647, 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.136 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.364643156, 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.138 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:31:01.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.141 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.142 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.143 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.002243825999997, 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.144 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.145 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:31:01.146 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.147 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.149 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.996992749, 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:01.150 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:31:01.152 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.153 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:01.154 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 73.996732378999994, 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:01.155 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.156 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:31:01.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:31:01.158 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:01.159 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:01.160 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:01.162 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:01.162+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:01.162 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:01.263 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:01.264+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:01.270 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:01.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:01.361 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:31:01.364 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:01.365+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:01.426 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:01.426 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:01.426 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:01.426 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:01.426 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:01.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:01.464 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:01.465+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:01.471 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:01.488 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:01.489 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:01.506 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:01.506 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:01.523 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:01.524 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:01.541 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:01.541 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:01.544 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:01.544 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:01.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:01.558 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:01.558 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:01.562 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:01.565 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:01.566+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:01.665 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:01.666 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:01.666 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:01.666+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:01.674 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:01.753 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:31:01.767 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:01.767+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:01.792 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:01.792 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:01.801 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:01.868 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:01.868+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:01.950 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:01.951 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:01.959 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:02.003 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:02.004+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:02.063 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.063 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.070 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:02.070+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:02.072 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:02.171 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:02.171+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:02.215 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.216 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:02.246 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:31:02.246 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:31:02.246 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-11T09:31:02.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-11T09:31:02.290 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:02.291+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.327 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:02.358 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:02.361 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:02.361 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:31:02.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-11T09:31:02.403 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:02.403+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:02.438 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.438 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:02.490 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:02.491+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:02.536 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:31:02.537 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:31:02.539 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:31:02.540 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:31:02.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-11T09:31:02.550 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.550 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.558 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:02.600 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:02.600+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:31:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:31:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:31:02.674 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:31:02.675 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:31:02.675 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:02.674+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:31:02.677 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:31:02.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-11T09:31:02.703 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.703 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.712 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:02.775 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:02.776+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:02.825 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.825 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.835 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:31:02.860 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:31:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:31:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:31:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:31:02.861 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:31:02.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:31:02.863 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:31:02.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-11T09:31:02.907 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:02.908+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:02.977 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:02.977+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:02.980 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:02.980 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:02.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:31:02.998 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:31:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:31:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:31:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:31:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:31:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:31:03.002 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:31:03.002 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:31:03.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-11T09:31:03.086 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:03.087+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:03.131 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:03.131 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:03.140 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:bench [] [] [] [] 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs debug_inject_read_zeros 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs files list 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:bluefs stats 2024-09-11T09:31:03.179 INFO:teuthology.orchestra.run.smithi017.stderr:bluestore allocator dump block 2024-09-11T09:31:03.180 INFO:teuthology.orchestra.run.smithi017.stderr:Error EINVAL: invalid command 2024-09-11T09:31:03.180 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:03.179+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:03.182 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:31:03.182 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-11T09:31:03.248 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:03.249 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:03.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:03.300 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:03.301+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:03.364 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:03.364 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:03.373 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:03.415 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:03.415+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:03.481 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:03.482+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:03.521 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:03.521 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:03.530 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:03.562 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:31:03.616 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:03.617+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:03.637 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:03.638 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:03.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:03.696 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:03.697 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:03.697 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:03.697+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:03.714 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:03.715 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:03.748 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:03.773 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:03.776 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:03.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:31:03.784 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:03.785+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:03.795 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:03.795 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:03.799 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:03.800 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:03.812 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:03.812 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:03.812 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:03.812 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:03.812 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:03.821 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:03.908 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:03.908+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:03.974 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782818247000002, 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:03.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:31:03.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:03.980 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782797528000003, 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:03.981 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:31:03.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782777009999997, 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:31:03.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:31:03.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:03.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:03.990 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782754353000001, 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:03.991 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:03.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782728630999998, 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:03.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:31:03.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:31:03.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:03.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782708544999998, 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:31:04.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.001 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:31:04.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.004 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782675019000003, 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.005 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:31:04.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:31:04.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782652048999999, 2024-09-11T09:31:04.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:04.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:04.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782631659000003, 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.014 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:31:04.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:31:04.018 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782611635000002, 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.019 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:31:04.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.022 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:31:04.023 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782592024000003, 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.024 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782566989999999, 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:31:04.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:31:04.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 54.782543072999999, 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.034 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:31:04.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:31:04.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:04.038 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.119151229000003, 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:04.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:31:04.041 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:31:04.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.119123618000003, 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.044 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.046 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:31:04.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:31:04.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.119102712999997, 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:31:04.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:04.054 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:31:04.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.119082366999997, 2024-09-11T09:31:04.056 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:04.057 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.058 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.059 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:31:04.060 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.118898545999997, 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.062 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.063 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:31:04.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.065 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 53.118868790999997, 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:04.068 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.069 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:31:04.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:04.071 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.072 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 16.895572292000001, 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:31:04.073 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:31:04.075 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.076 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:31:04.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:04.078 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:04.079 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:04.080 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:04.082 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:04.082 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:04.082+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:04.183 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:04.183+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:04.188 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:04.188 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:04.188 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:04.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:04.284 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:04.284+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:04.298 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:04.298 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:04.298 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:04.298 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:04.298 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:04.307 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:04.391 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:04.392+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:04.453 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 154.01534982699999, 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:31:04.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:31:04.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.461 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.399444099999997, 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.462 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:31:04.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:31:04.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.399246399000006, 2024-09-11T09:31:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:31:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:31:04.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:04.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.398987973000004, 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.477 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:31:04.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.482 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.398958089000004, 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.483 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:04.487 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.398424937000001, 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:04.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:31:04.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:31:04.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.397547488000001, 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:31:04.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.397508521999995, 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:31:04.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.500 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.397465045999994, 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:31:04.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:31:04.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:31:04.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.397340428999996, 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.505 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:31:04.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.396774958999998, 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.510 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:31:04.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.396744322999993, 2024-09-11T09:31:04.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:31:04.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.394607891000007, 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:31:04.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:04.524 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.393087402999996, 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:31:04.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.389013574000003, 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:31:04.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:04.534 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.388899780000003, 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:31:04.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:31:04.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:04.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.388815097999995, 2024-09-11T09:31:04.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:31:04.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:31:04.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.388769107000002, 2024-09-11T09:31:04.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:31:04.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.388397808999997, 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.551 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:31:04.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 77.388255860000001, 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:31:04.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:31:04.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:04.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:04.560 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:04.562 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:04.563 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:04.562+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:04.662 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:04.663+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:04.706 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:04.706 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:04.706 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:04.707 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:04.715 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:04.763 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:04.764+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:04.770 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:31:04.864 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:04.864+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:04.879 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:04.880 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:04.880 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:04.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:04.974 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:04.975+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:05.035 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.035 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695315827999998, 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:31:05.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:31:05.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.040 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695292608999999, 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.041 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:31:05.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695265448000001, 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:31:05.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695244922999997, 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:31:05.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.055 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695218490000002, 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:31:05.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:05.060 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695190338000003, 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:31:05.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:31:05.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695159957999998, 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.065 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:31:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695136884, 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695107522999997, 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:31:05.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:31:05.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695086297000003, 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:31:05.079 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:31:05.083 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695066003999997, 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.084 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:31:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695039956999999, 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:31:05.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:31:05.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.695018376, 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:31:05.093 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:31:05.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:31:05.097 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694997573000002, 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.098 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:31:05.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694978163000002, 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:31:05.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694941196000002, 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:31:05.107 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:31:05.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:31:05.111 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694908794, 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.112 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:05.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:31:05.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.116 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694886627000002, 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:31:05.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:31:05.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694861647000003, 2024-09-11T09:31:05.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 55.694841267999998, 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:31:05.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:31:05.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.131 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:05.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:05.133 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:05.132+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:05.232 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:05.233+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:05.276 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:05.276 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:05.276 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:05.285 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:05.334 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:05.334+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:05.427 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:05.427 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:05.427 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:05.427 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:05.427 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:05.434 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:05.435+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:05.436 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:05.535 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:05.536+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.581 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.597536198, 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.582 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.583 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:31:05.584 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.585 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:31:05.586 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.597490154, 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.587 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:05.588 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.589 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:31:05.590 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.591 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.59747848400001, 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:31:05.592 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.593 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:31:05.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:31:05.595 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.596 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4937 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 692224~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.59619021500001, 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033817248000000001, 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4937 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:05.598 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984467+0000", 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.05e-07 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.599 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984468+0000", 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1459999999999999e-06 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984471+0000", 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9979999999999999e-06 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984474+0000", 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6019999999999998e-06 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.600 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018208+0000", 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033734226999999999 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018247+0000", 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8858e-05 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018284+0000", 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7212000000000002e-05 2024-09-11T09:31:05.601 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.59615451400001, 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.602 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:05.603 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.604 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.605 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.606 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.596132477, 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.607 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:31:05.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:31:05.609 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.610 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.59609689200001, 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.611 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.613 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.614 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.615 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.596036549, 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.616 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.617 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:31:05.618 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.619 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.620 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.595987865, 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:31:05.621 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.622 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:31:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.623 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.624 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.625 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:31:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.626 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:31:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:31:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:31:05.627 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:31:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:31:05.628 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:31:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.629 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:31:05.631 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:31:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.632 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:31:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:31:05.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:31:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:31:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.634 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.635 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.595931401, 2024-09-11T09:31:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:31:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:31:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.637 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.639 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:31:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:31:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.640 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:31:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:31:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.641 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:31:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:31:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:31:05.642 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.643 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.59591184600001, 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.645 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.646 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:31:05.647 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.648 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.595864083, 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.650 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:31:05.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:31:05.652 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:05.653 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.595750345, 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.654 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:05.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.656 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.657 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:31:05.658 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.595731601, 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.661 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.663 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.664 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:31:05.665 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.666 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.667 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.59569261199999, 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.670 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.671 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.672 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.673 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.595669505, 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.674 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.675 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:31:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:31:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.677 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.678 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.887593014000004, 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:31:05.679 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.681 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:31:05.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.525193684000001, 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:31:05.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.687 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:31:05.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:05.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.519942607000004, 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.691 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:31:05.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.695 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 78.519682236999998, 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:05.696 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:31:05.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:05.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:31:05.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:31:05.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.702 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:05.702 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:05.702 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:05.703 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:05.704 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:05.704+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:05.801 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:31:05.803 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:05.804+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:05.833 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.833 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_repop(client.4358.0:28587 3.0 e138/76 3:01e2d9f4:::rbd_object_map.1106541f23ef:head v 138'1151, mlcod=138'1149)", 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:31:05.834057+0000", 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 0.000167057, 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018312800000000001, 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834057+0000", 2024-09-11T09:31:05.834 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834057+0000", 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834058+0000", 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8100000000000002e-07 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834063+0000", 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3090000000000003e-06 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834074+0000", 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.0878e-05 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.836 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834080+0000", 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1510000000000001e-06 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834129+0000", 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8418999999999998e-05 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:31:05.834151+0000", 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2742e-05 2024-09-11T09:31:05.837 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-11T09:31:05.838 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 1 2024-09-11T09:31:05.838 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:05.843 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:05.917 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:05.917 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:05.918+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:05.937 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:05.937 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:05.955 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:05.955 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:05.972 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:05.972 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:05.989 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:05.989 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:05.999 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:05.999 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.005 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:06.006+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:06.006 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:06.006 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:06.008 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:06.009 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:06.009 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:06.107 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:06.107+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:06.156 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.156 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:06.207 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:06.208+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:06.272 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.272 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:06.324 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:06.325+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:06.387 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.387 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.396 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:06.409 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:06.410+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:06.511 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:06.511+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:06.543 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.543 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.552 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:06.612 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:06.613+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:06.664 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.664 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:06.713 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:06.714+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:06.778 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.778 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.788 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:31:06.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:06.834 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:06.835+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:06.899 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:06.899 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:06.908 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:06.949 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:06.949+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:07.016 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:07.017+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:07.052 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.052 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:07.144 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:07.145+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:07.203 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.203 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:07.253 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:07.253+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:07.319 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:07.320+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:07.354 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.354 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.363 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:07.446 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:07.447+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:07.506 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.506 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.515 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:07.520 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:07.521+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:07.621 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:07.622+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:07.658 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.658 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:07.723 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:07.723+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:07.769 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.770 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.778 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:07.784 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:31:07.846 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:07.847+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:07.908 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:07.908 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:07.917 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:07.925 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:07.925+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:08.010 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:31:08.026 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:08.027+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:08.068 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:08.068 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:08.077 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:08.118 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:08.127 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:08.128+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:08.135 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:08.135 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:08.153 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:08.153 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:08.170 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:08.170 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:08.183 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:31:08.183 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:31:08.183 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-11T09:31:08.183 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.000000 2024-09-11T09:31:08.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0 2024-09-11T09:31:08.187 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:08.187 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:08.189 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:08.189 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:08.189 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:08.189 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:08.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.189 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.27771752, 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.190 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.191 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:31:08.192 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.193 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.277486948, 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.194 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.196 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:31:08.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:31:08.198 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.277389744, 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.199 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:08.200 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.201 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:31:08.202 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.203 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.27715498000001, 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:31:08.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.205 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:31:08.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.276958964, 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:31:08.208 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:31:08.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:31:08.211 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.276690527, 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:31:08.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.214 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:31:08.216 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.276588777, 2024-09-11T09:31:08.217 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:08.218 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:31:08.219 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.220 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:31:08.221 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.27619512, 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.222 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:08.223 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.224 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.225 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.226 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.27606887499999, 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.227 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:31:08.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:31:08.230 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.275859493, 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.231 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:31:08.233 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.234 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:31:08.235 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.275748544, 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.236 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:31:08.238 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.239 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.240 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.275492606, 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.241 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.242 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:31:08.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:31:08.244 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.275340077, 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.245 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:31:08.247 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.248 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.249 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.275147713, 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:08.250 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.251 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:31:08.252 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.253 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.274540965, 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.254 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.255 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:31:08.256 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.257 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:31:08.258 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.274487045, 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.259 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:08.260 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.261 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:31:08.262 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 117.274286476, 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.264 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:31:08.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:31:08.266 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:31:08.267 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 85.494665404000003, 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:31:08.268 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:31:08.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.271 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:31:08.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25886 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 22.11010164, 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97909465799999995, 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25886 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.275 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.276 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078566+0000", 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6339999999999999e-06 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078569+0000", 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4690000000000002e-06 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078577+0000", 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1160000000000001e-06 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078583+0000", 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078638+0000", 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4812000000000003e-05 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078645+0000", 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.686e-06 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057335+0000", 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97868998600000001 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057478+0000", 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014334499999999999 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057659+0000", 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018098100000000001 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.279 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25901 3.19 3:9f8aebfb:::rbd_data.11064477c615.0000000000000009:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.112921139000001, 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62440548799999995, 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:08.280 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25901 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.281 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075746+0000", 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4099999999999998e-07 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075750+0000", 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4599999999999996e-06 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075766+0000", 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5579000000000001e-05 2024-09-11T09:31:08.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075773+0000", 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6839999999999996e-06 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075846+0000", 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2606000000000002e-05 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075864+0000", 2024-09-11T09:31:08.283 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.766e-05 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.695967+0000", 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62010379199999999 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696129+0000", 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00016192800000000001 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696251+0000", 2024-09-11T09:31:08.284 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121199 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698192+0000", 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0019412120000000001 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700042+0000", 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001850317 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:08.285 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700107+0000", 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4455999999999994e-05 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700150+0000", 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3853999999999997e-05 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.286 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:08.288 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:08.288 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:08.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:08.289 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:08.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:08.290 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:08.288+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:08.388 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:08.389+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:08.437 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:08.437 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:08.437 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:08.437 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:08.446 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:08.490 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:08.490+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:08.552 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:08.553 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:08.553 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:08.553 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:08.553 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:08.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:08.591 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:08.592+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:08.670 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:08.670 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:08.670 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:08.670 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479479523999998, 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:31:08.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.672 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:31:08.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.674 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.675 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479458805, 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.676 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:31:08.678 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.679 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.680 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479438287000001, 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.681 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.682 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:31:08.683 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.684 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479415629999998, 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.685 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.686 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.687 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.688 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:31:08.689 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479389908000002, 2024-09-11T09:31:08.690 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:08.691 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.692 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:31:08.694 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479369822000002, 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.695 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:08.696 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.697 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:31:08.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.699 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479336296, 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.700 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.701 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:31:08.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:31:08.703 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479313326000003, 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.705 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.706 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:31:08.708 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:08.709 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479292936, 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.710 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.712 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:31:08.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.715 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479272911999999, 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.716 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.717 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.718 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:31:08.719 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479253301, 2024-09-11T09:31:08.720 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:08.721 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.722 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:31:08.724 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479228267000003, 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.725 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.726 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.727 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:31:08.728 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 59.479204350000003, 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:31:08.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.731 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:31:08.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.733 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:08.734 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 57.815812506, 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:08.735 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.736 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:31:08.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:31:08.738 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 57.815784895, 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:08.740 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.741 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:31:08.742 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:31:08.743 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:31:08.744 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 57.815763990000001, 2024-09-11T09:31:08.745 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.746 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.747 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.748 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:31:08.750 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 57.815743644000001, 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.751 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:08.752 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.753 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:31:08.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 57.815559823000001, 2024-09-11T09:31:08.756 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:08.757 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.758 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:31:08.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:31:08.760 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.761 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 57.815530068000001, 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:08.762 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.763 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:31:08.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:08.765 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.766 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 21.592233569000001, 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:08.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.768 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:31:08.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:31:08.770 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:08.771 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.772 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:31:08.773 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.774 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.774 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:08.774 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:08.774 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:08.775 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:08.776 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:08.775+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.3 primary-affinity to 0 (802) 2024-09-11T09:31:08.875 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:08.876+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:08.896 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:08.896 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:08.897 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:08.897 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:08.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:08.976 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:08.977+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:09.010 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:09.010 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:09.010 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:09.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:09.077 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:09.078+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:09.164 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:09.164 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:09.164 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 158.72642563599999, 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:31:09.165 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.166 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.167 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:31:09.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:09.169 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.170 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:31:09.171 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.172 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.173 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.110519909000004, 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:09.174 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:31:09.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.177 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:31:09.178 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:31:09.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.180 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.110322207999999, 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.182 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:31:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:09.184 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.185 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.186 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:31:09.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:09.188 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:09.189 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.110063781999997, 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:31:09.190 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:31:09.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.193 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:31:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:31:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:09.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.196 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.110033897999998, 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:31:09.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:09.201 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.109500745999995, 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:31:09.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.108623296999994, 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.206 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.210 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.108584331000003, 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:31:09.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.108540855000001, 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:31:09.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.219 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.108416238000004, 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:31:09.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.107850768000006, 2024-09-11T09:31:09.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:31:09.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.107820132000001, 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.229 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.231 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.233 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.1056837, 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:31:09.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.238 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.104163212000003, 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:31:09.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.100089382999997, 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:31:09.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.246 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.099975588999996, 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:31:09.248 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:31:09.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.251 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:31:09.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.099890907000002, 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:31:09.254 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:31:09.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:31:09.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.099844915999995, 2024-09-11T09:31:09.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:31:09.264 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.099473618000005, 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.265 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:31:09.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:09.269 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 82.099331668999994, 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:31:09.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.274 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:09.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:09.276 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:09.276+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:09.376 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:09.377+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:09.417 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:09.418 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:09.418 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:09.418 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:09.426 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:09.477 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:09.478+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:09.528 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:09.529 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:09.529 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:09.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:09.579 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:09.580+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:09.679 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:09.680+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:09.681 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:09.681 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342156121000002, 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.682 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.683 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.684 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.685 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.686 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342132902000003, 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.687 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.688 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.689 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:31:09.690 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.691 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342105740999997, 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:31:09.692 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.694 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.695 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:31:09.696 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342085216000001, 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.697 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.698 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.699 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:31:09.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342058782999999, 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:09.702 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.703 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:31:09.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.705 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342030631, 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.707 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:31:09.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.709 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.710 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.342000251000002, 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.711 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.713 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.714 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.715 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341977176999997, 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.716 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:31:09.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.718 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.719 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341947816000001, 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:31:09.720 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.721 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:09.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.723 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.724 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.34192659, 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.725 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:31:09.727 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.728 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.729 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341906297000001, 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.730 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:31:09.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.732 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.733 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341880250000003, 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.735 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:31:09.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:31:09.737 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.738 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341858668999997, 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.740 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:31:09.741 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:31:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341837865999999, 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.747 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:31:09.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.749 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.750 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341818455999999, 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.752 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:31:09.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:31:09.754 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:09.755 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341781488999999, 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:09.756 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.757 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.758 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:31:09.759 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341749086999997, 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.760 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.762 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.763 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.764 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341726919999999, 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.765 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:09.766 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.767 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:31:09.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.34170194, 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:31:09.769 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.770 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:31:09.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.772 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.773 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.341681561000001, 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:09.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.775 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:31:09.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:31:09.777 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.778 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:09.779 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:09.779 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:09.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:09.781 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:09.781+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:09.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:31:09.881 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:09.881+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:09.928 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:09.928 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:09.929 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:09.937 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:09.982 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:09.983+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:10.080 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:10.080 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:10.080 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:10.080 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:10.080 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:10.083 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:10.084+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:10.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:10.184 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:10.184+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:10.192 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:10.192 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.209135637, 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.193 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.195 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.197 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.209089593, 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:31:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.209077923, 2024-09-11T09:31:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:31:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:31:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.20775395299999, 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:31:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.211 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.207731916, 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.212 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.214 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.215 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.216 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.20769633099999, 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.217 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.218 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.219 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:31:10.220 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:10.221 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.20763598800001, 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.223 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:31:10.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.225 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.207587304, 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.227 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:31:10.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.229 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.20753084, 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.232 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:31:10.233 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.234 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.235 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.207511285, 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.236 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:10.237 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.238 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:31:10.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:10.240 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.207463522, 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.241 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.242 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.244 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.245 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.207349784, 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.246 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:31:10.247 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.248 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:31:10.249 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:10.250 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.20733104, 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.251 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.252 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:31:10.253 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.254 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.255 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.207292051, 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.256 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:31:10.257 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.258 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 119.20726894400001, 2024-09-11T09:31:10.260 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:10.261 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.262 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:31:10.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.264 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 100.49919245300001, 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.266 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:31:10.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.268 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.269 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.270 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.136793123000004, 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.271 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.272 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.273 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:31:10.274 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:10.275 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:31:10.276 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.131542046000007, 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.277 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:10.278 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:31:10.280 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.281 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 83.131281676, 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:10.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.283 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:31:10.284 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:31:10.285 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.286 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:31:10.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:29202 3.a 3:517570f5:::rbd_data.1106551ea4b0.0000000000000002:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:10.288 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 1.3139428470000001, 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92978124200000001, 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29202 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:10.289 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878315+0000", 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5999999999999999e-06 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.290 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878322+0000", 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878341+0000", 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9160999999999998e-05 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878369+0000", 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7441999999999999e-05 2024-09-11T09:31:10.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878447+0000", 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8467999999999995e-05 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878462+0000", 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4299e-05 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:10.292 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804505+0000", 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92604313900000002 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804629+0000", 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012435199999999999 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804779+0000", 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014989300000000001 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.293 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.807759+0000", 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029804020000000001 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808015+0000", 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025595300000000001 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808055+0000", 2024-09-11T09:31:10.294 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9987000000000002e-05 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808095+0000", 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9329999999999998e-05 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:10.295 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:10.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:31:10.297 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:10.297+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:10.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:10.397 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:10.397+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:10.405 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:10.408 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:10.408 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:10.409 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:10.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:10.423 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:10.424 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:10.442 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:10.442 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:10.459 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:10.459 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:10.476 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:10.493 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:10.496 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:10.531 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:10.532 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:10.532 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:10.532+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:10.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:10.624 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:10.625+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:10.688 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:10.688 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:10.696 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:10.699 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:10.700+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:10.799 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:31:10.800 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:10.800+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:10.855 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:10.856 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:10.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:10.905 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:10.906+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:11.001 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:11.001+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:11.015 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.015 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.024 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:11.109 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:11.110+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:11.171 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.172 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:11.224 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:11.225+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:11.283 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.283 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.292 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:11.333 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:11.334+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:11.404 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:11.405+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:11.435 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.435 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.443 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:11.527 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:11.528+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:11.586 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.586 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.595 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:11.636 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:11.636+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:11.706 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:11.707+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:11.738 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.739 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.747 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:11.831 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:11.832+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:11.897 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:11.897 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:11.907 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:11.949 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:11.949+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:12.009 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:12.010+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:12.053 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:12.053 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:12.062 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:12.110 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:12.111+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:12.205 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:12.205 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:12.212 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:12.212+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:12.213 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:12.312 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:12.313+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:12.315 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:12.315 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:12.324 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:12.413 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:12.414+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:12.467 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:12.468 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:12.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:12.497 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:31:12.513 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:12.514+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:12.600 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:12.600 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:12.610 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:12.636 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:12.637+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:12.653 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:12.653 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:12.671 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:12.671 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:12.688 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:12.688 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:12.705 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:12.706 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:12.716 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:12.717+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:12.721 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:12.722 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:12.722 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:12.722 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:12.722 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:12.722 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:12.723 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:12.727 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:12.727 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:12.731 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:12.817 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:12.818+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:12.820 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-11T09:31:12.886 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97524787499999, 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.887 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.888 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.975017303, 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.892 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.894 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.974920099, 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:31:12.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:31:12.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.974685335, 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.901 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:31:12.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.904 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.974489319, 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.906 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:31:12.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.910 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.974220882, 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:31:12.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.913 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.974119132, 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.915 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:31:12.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.918 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.919 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97372547499999, 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.920 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:31:12.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.924 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97359923, 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:31:12.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:31:12.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.928 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.973389848, 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.930 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:31:12.932 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97327889899999, 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.934 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.935 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:31:12.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.937 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.938 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.973022961, 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.939 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:31:12.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.942 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97287043199999, 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:31:12.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:31:12.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97267806799999, 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:31:12.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:31:12.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.97207132, 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:31:12.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.956 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.9720174, 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:31:12.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:31:12.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:31:12.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 121.971816831, 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.961 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:31:12.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.965 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 90.192195759000001, 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.966 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:31:12.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:31:12.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.971 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25886 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 26.807631995000001, 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97909465799999995, 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25886 2024-09-11T09:31:12.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078566+0000", 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6339999999999999e-06 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078569+0000", 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4690000000000002e-06 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078577+0000", 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1160000000000001e-06 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078583+0000", 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078638+0000", 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4812000000000003e-05 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078645+0000", 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.686e-06 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057335+0000", 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97868998600000001 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057478+0000", 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014334499999999999 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057659+0000", 2024-09-11T09:31:12.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018098100000000001 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25901 3.19 3:9f8aebfb:::rbd_data.11064477c615.0000000000000009:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 25.810451493999999, 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62440548799999995, 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:12.977 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25901 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:12.978 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075746+0000", 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4099999999999998e-07 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075750+0000", 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4599999999999996e-06 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:12.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075766+0000", 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5579000000000001e-05 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075773+0000", 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6839999999999996e-06 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075846+0000", 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2606000000000002e-05 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075864+0000", 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.766e-05 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.695967+0000", 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62010379199999999 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696129+0000", 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00016192800000000001 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696251+0000", 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121199 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698192+0000", 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0019412120000000001 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700042+0000", 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001850317 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700107+0000", 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4455999999999994e-05 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700150+0000", 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3853999999999997e-05 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:12.983 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:12.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:12.985 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:12.985+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:13.086 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:13.086+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:13.092 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:13.092 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:13.092 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:13.093 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:13.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:13.186 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:13.187+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:13.250 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:13.250 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:13.250 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:13.250 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:13.250 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:13.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:13.287 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:13.288+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:13.367 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.176004688999996, 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.368 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.369 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:31:13.370 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.371 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175983970000004, 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.372 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:31:13.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175963452000005, 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.377 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.378 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:31:13.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175940795000002, 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:31:13.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.383 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175915072999999, 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.386 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:31:13.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.390 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175894987000007, 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.391 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.392 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:31:13.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175861460999997, 2024-09-11T09:31:13.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:13.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.397 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:31:13.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175838490999993, 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.400 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:31:13.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.404 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175818101000004, 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.405 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.407 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175798076999996, 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:13.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:31:13.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175778465999997, 2024-09-11T09:31:13.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:31:13.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:31:13.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.419 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175753431999993, 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:31:13.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:31:13.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 64.175729515, 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.424 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:31:13.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:31:13.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 62.512337670999997, 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.429 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:13.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:31:13.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:13.434 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 62.512310059999997, 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:13.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:31:13.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.439 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 62.512289154999998, 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:13.440 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.441 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:31:13.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:13.445 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 62.512268808999998, 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:31:13.450 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 62.512084987999998, 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:13.451 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:31:13.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:13.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 62.512055232999998, 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:31:13.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:31:13.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:13.461 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 26.288758734000002, 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.463 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:31:13.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:13.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:13.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:13.470 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:13.470 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:13.470+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:13.571 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:13.571+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:13.576 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:13.576 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:13.577 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:13.585 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:13.671 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:13.672+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:13.728 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:13.728 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:13.729 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:13.729 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:13.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:13.772 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:13.773+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:13.804 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:31:13.805 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:31:13.805 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-11T09:31:13.805 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-11T09:31:13.807 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is set 2024-09-11T09:31:13.853 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:13.853 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 163.41518981300001, 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.854 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.855 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.856 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:13.857 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.858 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:31:13.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.799284086, 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:31:13.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.863 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.864 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:31:13.865 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.866 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.799086384999995, 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.868 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:31:13.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.870 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.871 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:13.872 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.798827958999993, 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.876 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:31:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:13.878 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.798798074999993, 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:31:13.880 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.882 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:31:13.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.884 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.798264923000005, 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.886 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:31:13.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:31:13.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.889 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.797387474000004, 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:31:13.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.891 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.892 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:31:13.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.894 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.797348507999999, 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.895 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.896 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.897 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:31:13.898 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:31:13.899 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.797305031999997, 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.902 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.903 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:31:13.904 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.905 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.797180415, 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.906 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.907 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:31:13.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:31:13.910 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.796614945000002, 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.911 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:13.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:31:13.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.916 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.796584308999996, 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:31:13.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:31:13.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.794447876999996, 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:31:13.921 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:31:13.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:31:13.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.792927388999999, 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.926 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.930 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.788853560000007, 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:31:13.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.788739766000006, 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:31:13.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:31:13.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:31:13.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.788655083999998, 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:31:13.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.948 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.788609093000005, 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.949 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:31:13.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:31:13.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.788237795000001, 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:31:13.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 86.788095846000004, 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:13.959 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:13.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:31:13.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:31:13.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:13.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:13.964 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:13.965 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:13.966 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:13.967 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:13.967 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:13.966+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:13.967 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-09-11T09:31:13.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-11T09:31:14.066 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:14.067+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:14.106 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-11T09:31:14.111 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:14.111 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:14.111 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:14.119 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:14.203 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:14.204+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:14.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-11T09:31:14.263 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:14.264 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:14.264 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:14.268 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:14.269+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:14.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:14.369 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:14.370+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:14.380 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-11T09:31:14.417 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.417 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077435291, 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.418 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.419 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.420 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:31:14.421 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.422 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077412072000001, 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.423 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.424 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:31:14.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:31:14.426 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.427 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077384910999996, 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.428 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.430 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:31:14.431 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.432 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077364385999999, 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:31:14.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.434 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:31:14.435 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077337952999997, 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.438 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.439 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.440 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:31:14.441 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.442 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077309800999998, 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.443 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.444 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:31:14.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.446 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077279421, 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:14.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:31:14.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:31:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:31:14.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077256347000002, 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:31:14.454 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:14.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:31:14.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077226985999999, 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:14.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.462 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.463 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077205759999998, 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.464 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:31:14.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.468 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077185467000007, 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:31:14.469 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:14.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:31:14.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077159420000001, 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:31:14.474 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:31:14.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077137839000002, 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:31:14.479 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:31:14.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.484 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077117036000004, 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:31:14.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.489 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077097625999997, 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.490 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:31:14.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:14.496 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077060658999997, 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:31:14.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:31:14.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.502 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077028256999995, 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.503 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:31:14.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:31:14.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.077006089999998, 2024-09-11T09:31:14.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:14.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:31:14.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.076981110000006, 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.515 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:31:14.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 65.076960731, 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:31:14.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.523 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.524 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:14.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-11T09:31:14.526 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:14.526+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:14.527 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:14.626 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:14.627+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:14.669 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:14.669 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:14.669 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:14.678 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:14.727 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:14.728+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:14.728 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:31:14.781 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:14.782 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:14.782 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:14.790 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:14.831 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:14.832+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:14.854 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:14.875 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:14.875 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:14.893 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:14.893 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:14.912 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:14.913 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:14.929 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:14.929+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:14.930 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:14.931 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:14.939 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:14.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.95621006, 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.940 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:31:14.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.956164016, 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.945 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:31:14.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:31:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.956152346, 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:31:14.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.954 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.95482837599999, 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:31:14.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:31:14.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954806339, 2024-09-11T09:31:14.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:14.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:31:14.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.95477075399999, 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:31:14.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.95471041099999, 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:31:14.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:31:14.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954661727, 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:31:14.973 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:31:14.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:31:14.977 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954605263, 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.978 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:31:14.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954585708, 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:31:14.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:31:14.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.987 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954537945, 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.988 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:31:14.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954424207, 2024-09-11T09:31:14.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:31:14.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954405463, 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:14.997 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:14.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:14.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.954366474, 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:31:15.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:31:15.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 123.95434336700001, 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:31:15.006 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:31:15.010 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 105.24626687600001, 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.011 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:15.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:31:15.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:15.016 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.883867546000005, 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:15.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.018 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:31:15.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:15.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:15.022 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.878616468999994, 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:31:15.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 87.878356099000001, 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:15.027 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:31:15.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:31:15.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:29202 3.a 3:517570f5:::rbd_data.1106551ea4b0.0000000000000002:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 6.0610172699999998, 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92978124200000001, 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:15.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29202 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878315+0000", 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5999999999999999e-06 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878322+0000", 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2024-09-11T09:31:15.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878341+0000", 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9160999999999998e-05 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878369+0000", 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7441999999999999e-05 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.036 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878447+0000", 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8467999999999995e-05 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878462+0000", 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4299e-05 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804505+0000", 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92604313900000002 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804629+0000", 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012435199999999999 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804779+0000", 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014989300000000001 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.807759+0000", 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029804020000000001 2024-09-11T09:31:15.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808015+0000", 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025595300000000001 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808055+0000", 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9987000000000002e-05 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:15.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808095+0000", 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9329999999999998e-05 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:15.040 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:15.041 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:15.042 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:15.042 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:15.042+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:15.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:15.043 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:15.043 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:15.143 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:15.143+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:15.191 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:15.191 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:15.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:15.243 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:15.244+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:15.343 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:15.344+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:15.346 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:15.346 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:15.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:15.444 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:15.445+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:15.462 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:15.462 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:15.470 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:15.554 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:15.555+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:15.618 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:15.618 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:15.627 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:15.646 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:15.647+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:15.747 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:15.748+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:15.774 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:15.774 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:15.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:15.829 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:31:15.875 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:15.875+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:15.899 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:15.899 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:15.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:15.954 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:15.955+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:16.022 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.023 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.031 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:16.073 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:16.073+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:16.151 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:16.151+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:16.174 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.174 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.183 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:16.252 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:16.253+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:16.286 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.286 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:16.353 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:16.354+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:16.397 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.397 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:16.454 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:16.455+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:16.507 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.508 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:16.555 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:16.555+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:16.619 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.619 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:16.671 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:16.672+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:16.731 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.731 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:16.781 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:16.781+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:16.831 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:31:16.857 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:16.858+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:16.894 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:16.894 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:16.904 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:16.958 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:16.959+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:17.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:31:17.046 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:17.047 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:17.055 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:17.059 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:17.059+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:17.154 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:17.159 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:17.160+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:17.171 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:17.172 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:17.189 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:17.198 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:17.198 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:17.198 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:17.198 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:17.205 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:17.205 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:17.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:17.222 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:17.222 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:17.239 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:17.239 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:17.242 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:17.243 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:17.261 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:17.261+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:17.329 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:17.329 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:17.329 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:17.329 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:17.330 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:17.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:17.381 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:17.382+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:17.447 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:17.447 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:17.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.535824619, 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.448 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:31:17.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.535594047, 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:31:17.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:31:17.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:31:17.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.535496843, 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:31:17.458 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.53526207900001, 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.463 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.467 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.535066063, 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.468 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:31:17.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.534797626, 2024-09-11T09:31:17.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:31:17.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:31:17.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.476 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.534695876, 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:31:17.477 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.478 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.479 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:31:17.480 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.481 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.534302219, 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.482 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.483 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:31:17.484 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.485 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:17.486 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.53417597399999, 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:17.487 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.488 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:31:17.489 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.490 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.533966592, 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.491 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.492 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.493 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:31:17.494 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.533855643, 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.496 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.497 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.498 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.500 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.533599705, 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.501 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:31:17.502 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.503 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:31:17.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.533447176, 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.505 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.506 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:31:17.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.508 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.509 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.533254812, 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.510 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:31:17.512 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.513 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.514 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.532648064, 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.515 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:31:17.516 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.517 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.518 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.532594144, 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.519 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.521 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:31:17.523 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 126.532393575, 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.524 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.525 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:31:17.526 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:31:17.528 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 94.752772503000003, 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.529 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:17.530 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.531 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:31:17.532 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.533 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.534 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25886 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 31.368208739, 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97909465799999995, 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25886 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.536 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078566+0000", 2024-09-11T09:31:17.537 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6339999999999999e-06 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078569+0000", 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4690000000000002e-06 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078577+0000", 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1160000000000001e-06 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078583+0000", 2024-09-11T09:31:17.538 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078638+0000", 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4812000000000003e-05 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078645+0000", 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.686e-06 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.539 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057335+0000", 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97868998600000001 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057478+0000", 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014334499999999999 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057659+0000", 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018098100000000001 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25901 3.19 3:9f8aebfb:::rbd_data.11064477c615.0000000000000009:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.371028238000001, 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62440548799999995, 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:17.541 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25901 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075746+0000", 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4099999999999998e-07 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075750+0000", 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4599999999999996e-06 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075766+0000", 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5579000000000001e-05 2024-09-11T09:31:17.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075773+0000", 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6839999999999996e-06 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075846+0000", 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2606000000000002e-05 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:17.544 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075864+0000", 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.766e-05 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.695967+0000", 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62010379199999999 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696129+0000", 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00016192800000000001 2024-09-11T09:31:17.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696251+0000", 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121199 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698192+0000", 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0019412120000000001 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:17.546 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700042+0000", 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001850317 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700107+0000", 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4455999999999994e-05 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700150+0000", 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3853999999999997e-05 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.547 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.548 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.548 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:17.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:17.550 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:17.550+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:17.649 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:17.650+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:17.697 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:17.698 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:17.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:17.698 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:17.707 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:17.750 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:17.751+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:17.813 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:17.814 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:17.814 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:17.814 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:17.814 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:17.823 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:17.851 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:17.852+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:17.946 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:17.946 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:17.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:17.946 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:17.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.755046680999996, 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:17.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.755025962000005, 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.952 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:31:17.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.953 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:31:17.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:31:17.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.956 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.755005444000005, 2024-09-11T09:31:17.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:17.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.961 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754982787000003, 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.962 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:31:17.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.966 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754957064999999, 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:17.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.972 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754936979000007, 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:31:17.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:31:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754903452999997, 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.978 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:31:17.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:17.982 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754880482999994, 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:17.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:31:17.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754860093000005, 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.987 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.991 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754840068999997, 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.992 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:31:17.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:31:17.995 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754820457999998, 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:17.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:17.997 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:17.998 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:17.999 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.000 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754795423999994, 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.001 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:31:18.002 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.003 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.004 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 68.754771507000001, 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.005 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.006 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:31:18.008 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.009 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.091379662999998, 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.011 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:31:18.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.013 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:31:18.014 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.015 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.091352052000005, 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.016 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.017 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:31:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.091331147000005, 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:31:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:31:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.091310801000006, 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:31:18.031 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.032 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.091126979999999, 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.033 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.034 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.035 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.036 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:31:18.037 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 67.091097224999999, 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:18.039 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.042 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.044 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.045 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 30.867800725999999, 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:18.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:18.048 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.049 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:31:18.050 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:18.051 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:18.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:18.053 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:18.055 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:18.055 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:18.055+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:18.156 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:18.156+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:18.164 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:18.164 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:18.164 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:18.164 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:18.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:18.257 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:18.258+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:18.317 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:18.317 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:18.318 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:18.318 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:18.318 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:18.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:18.366 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:18.367+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:18.459 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:18.459+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:31:18.471 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 168.034044631, 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:31:18.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:31:18.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.476 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:31:18.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.418138904000003, 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:31:18.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:31:18.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.484 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.417941202999998, 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.487 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.417682776999996, 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:18.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:31:18.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:18.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.417652892999996, 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:31:18.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:31:18.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.417119740999993, 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:31:18.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:31:18.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.505 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.416242292000007, 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:31:18.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:18.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.416203326000002, 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:18.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:31:18.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:31:18.514 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.41615985, 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.515 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.519 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.416035233000002, 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:31:18.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:31:18.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:31:18.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.415469763000004, 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.527 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.415439126999999, 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.530 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:31:18.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:31:18.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.534 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.413302694999999, 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.535 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:18.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:31:18.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.411782207000002, 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:31:18.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:31:18.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:31:18.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.544 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.407708377999995, 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.548 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:31:18.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.407594583999995, 2024-09-11T09:31:18.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:18.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:31:18.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:18.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.407509902000001, 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:31:18.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:31:18.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:31:18.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.407463910999994, 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.565 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.566 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:31:18.567 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.568 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.407092613000003, 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.569 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.570 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:31:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.573 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 91.406950663999993, 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:18.574 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:31:18.575 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.576 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:31:18.577 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:18.578 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:18.580 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:18.580 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:18.580+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:18.680 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:18.681+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:18.723 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:18.723 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:18.723 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:18.732 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:18.781 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:18.782+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:18.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:31:18.882 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:18.883+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:18.884 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:18.885 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:18.885 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:18.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:18.983 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:18.984+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703328158000005, 2024-09-11T09:31:19.043 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:19.044 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.045 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:31:19.046 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.047 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703304939000006, 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:31:19.048 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.050 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:31:19.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.052 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703277778, 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.054 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:19.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.056 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.057 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703257253000004, 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.058 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:19.059 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.060 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:31:19.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.062 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703230820000002, 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.063 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.064 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.065 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.066 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.067 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703202668000003, 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:31:19.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.069 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:31:19.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.071 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.072 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703172288000005, 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.073 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:31:19.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.075 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.076 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703149214000007, 2024-09-11T09:31:19.077 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.078 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:31:19.079 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.080 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.081 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703119853000004, 2024-09-11T09:31:19.082 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:19.083 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.084 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:31:19.085 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.086 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703098627000003, 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.087 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.088 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:31:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:31:19.090 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.091 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703078333999997, 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.094 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.095 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.096 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703052287000006, 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:31:19.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.098 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:31:19.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.100 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.101 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703030706000007, 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.102 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:31:19.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.104 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:31:19.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.703009902999995, 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:31:19.106 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.107 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.108 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:31:19.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.110 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.702990493000001, 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.111 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:19.112 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.113 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.114 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.115 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.702953526000002, 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.117 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:31:19.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.119 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.120 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.702921124, 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.121 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.123 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.124 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.125 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.702898957000002, 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.126 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.128 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:31:19.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:31:19.130 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.702873976999996, 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.131 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.132 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.133 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.134 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 69.702853598000004, 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:19.136 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.137 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:19.138 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.139 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.140 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:19.141 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:19.141 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:19.142 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:19.142+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:19.142 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:19.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-11T09:31:19.244 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:19.244+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:19.246 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:19.246 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:19.246 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:19.246 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:19.255 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:19.344 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:19.345+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:19.351 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:19.357 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:19.357 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:19.358 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:19.358 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:19.366 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:19.367 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:19.368 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:19.385 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:19.385 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:19.402 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:19.402 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:19.418 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:19.436 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:19.436 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:19.439 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:19.440 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:19.445 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:19.446+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:19.510 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:19.510 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:19.518 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:19.546 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:19.547+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:19.626 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:19.627 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:19.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:19.647 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:19.648+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:19.703 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 3, 2] out_osds: [] dead_osds: [0] live_osds: [1, 2, 3, 4, 5] 2024-09-11T09:31:19.703 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-11T09:31:19.704 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 4, 5, 3, 2] 2024-09-11T09:31:19.704 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-11T09:31:19.743 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:19.743 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:19.747 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:19.748+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:19.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:19.844 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:31:19.849 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:19.849+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:19.916 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:19.916 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:19.927 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:19.971 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:19.972+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:20.036 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-11T09:31:20.036+0000 7fef02752640 -1 mon.a@0(leader).osd e152 definitely_dead 0 2024-09-11T09:31:20.038 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.039 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.048 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:20.050 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:20.051+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:20.151 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:20.152+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:20.195 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.195 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.203 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:20.252 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:20.253+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:20.311 INFO:teuthology.orchestra.run.smithi017.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.311 INFO:teuthology.orchestra.run.smithi017.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:20.354 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:20.355+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:20.423 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.423 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.432 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:20.475 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:20.476+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:20.535 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.536 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.545 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:20.585 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:20.586+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:20.656 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:20.657+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:20.686 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.687 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:20.757 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:20.757+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:20.799 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.799 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:20.854 INFO:teuthology.orchestra.run.smithi017.stderr:marked out osd.3. 2024-09-11T09:31:20.859 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:20.859+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:20.914 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:20.914 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:20.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:20.959 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:20.960+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:21.060 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:21.061+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:21.067 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:21.067 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:21.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:21.161 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:21.161+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:21.179 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:21.180 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:21.188 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:21.272 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:21.273+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:21.331 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:21.332 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:21.340 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:21.363 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:21.363+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:21.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-11T09:31:21.482 INFO:teuthology.orchestra.run.smithi096.stdout:ERROR: (22) Invalid argument 2024-09-11T09:31:21.483 INFO:teuthology.orchestra.run.smithi096.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-11T09:31:21.485 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:21.486+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:21.491 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T09:31:21.562 INFO:teuthology.orchestra.run.smithi017.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-11T09:31:21.564 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:21.565+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:21.579 INFO:teuthology.orchestra.run.smithi017.stdout:osd.1: {} 2024-09-11T09:31:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-11T09:31:21.598 INFO:teuthology.orchestra.run.smithi017.stdout:osd.2: {} 2024-09-11T09:31:21.599 INFO:teuthology.orchestra.run.smithi017.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-11T09:31:21.609 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:21.609 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:21.609 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:21.609 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:21.616 INFO:teuthology.orchestra.run.smithi017.stdout:osd.3: {} 2024-09-11T09:31:21.616 INFO:teuthology.orchestra.run.smithi017.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-11T09:31:21.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-11T09:31:21.634 INFO:teuthology.orchestra.run.smithi017.stdout:osd.4: {} 2024-09-11T09:31:21.634 INFO:teuthology.orchestra.run.smithi017.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-11T09:31:21.651 INFO:teuthology.orchestra.run.smithi017.stdout:osd.5: {} 2024-09-11T09:31:21.651 INFO:teuthology.orchestra.run.smithi017.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-11T09:31:21.655 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-11T09:31:21.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-11T09:31:21.697 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:21.698+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:21.726 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:21.727 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:21.727 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:21.727 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:21.727 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:21.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-11T09:31:21.776 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-11T09:31:21.777+0000 7f4f39152640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32194) UID: 0 2024-09-11T09:31:21.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-11T09:31:21.866 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:21.867+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:21.888 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3927 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 210944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97729775400001, 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038932054000000001, 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3927 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.890 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910949+0000", 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.58e-07 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910951+0000", 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4729999999999999e-06 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.891 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910956+0000", 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9389999999999996e-06 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.910960+0000", 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.365e-06 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949686+0000", 2024-09-11T09:31:21.892 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038725945999999997 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949763+0000", 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.7134999999999999e-05 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949881+0000", 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000117738 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3933 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 221184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97706718200001, 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038914569000000003, 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.894 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3933 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911179+0000", 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911180+0000", 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9399999999999995e-07 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911182+0000", 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4140000000000001e-06 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911187+0000", 2024-09-11T09:31:21.896 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0880000000000002e-06 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911191+0000", 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.489e-06 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949914+0000", 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038722964999999998 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.897 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.949997+0000", 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.3786e-05 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950094+0000", 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6333000000000001e-05 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.898 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3935 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 223232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.976969978, 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039028753999999999, 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3935 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.899 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911276+0000", 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911277+0000", 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2599999999999998e-07 2024-09-11T09:31:21.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911279+0000", 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.216e-06 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911284+0000", 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6369999999999998e-06 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911287+0000", 2024-09-11T09:31:21.901 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1750000000000001e-06 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950124+0000", 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038837346000000002 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950206+0000", 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2249999999999993e-05 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.902 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950305+0000", 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8703999999999996e-05 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3940 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 200704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.976735214, 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039009528000000002, 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.903 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3940 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911511+0000", 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911512+0000", 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6499999999999999e-07 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.905 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911514+0000", 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2199999999999999e-06 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911519+0000", 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8019999999999999e-06 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911522+0000", 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3160000000000001e-06 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950337+0000", 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038814792000000001 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950419+0000", 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2645000000000006e-05 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950521+0000", 2024-09-11T09:31:21.907 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000101288 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3944 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 212992~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97653919800001, 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039020171999999999, 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.908 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3944 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.909 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911707+0000", 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911708+0000", 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6100000000000001e-07 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911710+0000", 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6350000000000002e-06 2024-09-11T09:31:21.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911715+0000", 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8250000000000004e-06 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911718+0000", 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2600000000000001e-06 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.911 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950551+0000", 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038832943000000002 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950631+0000", 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.0006999999999995e-05 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950727+0000", 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.6040999999999997e-05 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.912 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3950 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 225280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97627076099999, 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038978693000000002, 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.913 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3950 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911975+0000", 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911976+0000", 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8299999999999996e-07 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911979+0000", 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.0199999999999999e-06 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911985+0000", 2024-09-11T09:31:21.915 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9599999999999997e-06 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.911989+0000", 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.8190000000000002e-06 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950759+0000", 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038769574000000001 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.916 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950846+0000", 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.7535999999999997e-05 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950954+0000", 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000108101 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.917 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3951 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 202752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.976169011, 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039069395, 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3951 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.918 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912077+0000", 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.919 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912078+0000", 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6199999999999997e-07 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912081+0000", 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8329999999999998e-06 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912086+0000", 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.057e-06 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912089+0000", 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.563e-06 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.950992+0000", 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038902269000000003 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951039+0000", 2024-09-11T09:31:21.921 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7398999999999998e-05 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951147+0000", 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000107612 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3959 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 204800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:21.922 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97577535400001, 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038912574999999998, 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3959 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.923 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912471+0000", 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912472+0000", 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:21.924 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912475+0000", 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.005e-06 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912480+0000", 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0889999999999999e-06 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.925 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912483+0000", 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4359999999999998e-06 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951189+0000", 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038706305000000003 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951278+0000", 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8677999999999994e-05 2024-09-11T09:31:21.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951383+0000", 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00010541 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3962 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 217088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97564910899999, 2024-09-11T09:31:21.927 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038945510000000003, 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3962 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912597+0000", 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912598+0000", 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5400000000000001e-07 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.929 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912600+0000", 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6620000000000001e-06 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912606+0000", 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0839999999999999e-06 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912609+0000", 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5420000000000001e-06 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951417+0000", 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038808238000000002 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951473+0000", 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5269999999999998e-05 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951543+0000", 2024-09-11T09:31:21.931 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0060000000000003e-05 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3964 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 215040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97543972700001, 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038968651999999999, 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.932 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3964 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.933 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912807+0000", 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912810+0000", 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6199999999999999e-06 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.934 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912815+0000", 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2179999999999998e-06 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912819+0000", 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4800000000000001e-06 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.935 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951576+0000", 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038757891000000003 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951670+0000", 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3555999999999997e-05 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951775+0000", 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000105235 2024-09-11T09:31:21.936 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3968 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 227328~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97532877800001, 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039045135000000002, 2024-09-11T09:31:21.937 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3968 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.938 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912917+0000", 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912918+0000", 2024-09-11T09:31:21.939 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912921+0000", 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.8100000000000002e-06 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912927+0000", 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2809999999999997e-06 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.940 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.912931+0000", 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7560000000000001e-06 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951813+0000", 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038882103000000001 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951871+0000", 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7451000000000003e-05 2024-09-11T09:31:21.941 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951963+0000", 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.2064000000000005e-05 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3973 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 425984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97507284, 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038928294000000002, 2024-09-11T09:31:21.942 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3973 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.943 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913173+0000", 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913174+0000", 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.61e-07 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913177+0000", 2024-09-11T09:31:21.944 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.7640000000000001e-06 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913182+0000", 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1139999999999997e-06 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913185+0000", 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.41e-06 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.945 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.951993+0000", 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038807531999999999 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952043+0000", 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9623999999999998e-05 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952102+0000", 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9188999999999999e-05 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.946 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3977 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 438272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97492031100001, 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.039003165999999999, 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.947 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3977 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.948 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913326+0000", 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913329+0000", 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2189999999999998e-06 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.949 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913336+0000", 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.8419999999999999e-06 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913339+0000", 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.455e-06 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952139+0000", 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038800117000000002 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.950 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952226+0000", 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.6749000000000006e-05 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952329+0000", 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000103294 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.951 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3979 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 206848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97472794699999, 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038936529999999997, 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3979 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.952 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913518+0000", 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913519+0000", 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5199999999999996e-07 2024-09-11T09:31:21.953 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913522+0000", 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6390000000000001e-06 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913527+0000", 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0350000000000001e-06 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.913531+0000", 2024-09-11T09:31:21.954 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.5029999999999999e-06 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952358+0000", 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038826468000000003 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952401+0000", 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3129e-05 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.955 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952455+0000", 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4104000000000002e-05 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3992 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 655360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.974121199, 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038849966, 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.956 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3992 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.957 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914125+0000", 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914126+0000", 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.2699999999999998e-07 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914129+0000", 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.2349999999999999e-06 2024-09-11T09:31:21.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914139+0000", 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.7380000000000006e-06 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914144+0000", 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1159999999999998e-06 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.959 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952818+0000", 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038673781999999997 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952909+0000", 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.1217000000000004e-05 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952975+0000", 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6050999999999995e-05 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.960 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3995 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.974067279, 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038905517000000001, 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.961 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3995 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914179+0000", 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914180+0000", 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9599999999999999e-07 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914183+0000", 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.32e-06 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914193+0000", 2024-09-11T09:31:21.963 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9170000000000001e-06 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914198+0000", 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.378e-06 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.952989+0000", 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038791006000000003 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.964 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953029+0000", 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0080999999999998e-05 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953084+0000", 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5118999999999999e-05 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4756.0:3999 3.0 3:01610940:::rbd_data.11069296af76.0000000000000000:b [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 130.97386671000001, 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038910710000000001, 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4756", 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 3999 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.966 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.967 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914380+0000", 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4700000000000001e-07 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914384+0000", 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0409999999999999e-06 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914393+0000", 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.8279999999999992e-06 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.968 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.914399+0000", 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2040000000000002e-06 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953113+0000", 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.038713145999999997 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953191+0000", 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.8109999999999996e-05 2024-09-11T09:31:21.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:10.953290+0000", 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.9634000000000002e-05 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5033.0:2169 3.4 3:277cbcab:::rbd_object_map.110637f3dd8a:head [call lock.assert_locked in=17b,call] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 99.194245637999998, 2024-09-11T09:31:21.970 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.003158411, 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5033", 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/3085765473", 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 2169 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:21.971 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694001+0000", 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694003+0000", 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0360000000000001e-06 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.972 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694010+0000", 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1439999999999997e-06 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694024+0000", 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.3767e-05 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694033+0000", 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.8490000000000001e-06 2024-09-11T09:31:21.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694104+0000", 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.0140000000000003e-05 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:42.694130+0000", 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.6829e-05 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.974 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692712+0000", 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.99858130599999995 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.692933+0000", 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000221595 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.693231+0000", 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00029749600000000001 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.694136+0000", 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00090479500000000001 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697047+0000", 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00291129 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697112+0000", 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.5366999999999999e-05 2024-09-11T09:31:21.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:29:43.697159+0000", 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6796999999999999e-05 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25886 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118)", 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.809681873999999, 2024-09-11T09:31:21.977 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97909465799999995, 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25886 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.978 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078564+0000", 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078566+0000", 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.6339999999999999e-06 2024-09-11T09:31:21.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078569+0000", 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4690000000000002e-06 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078577+0000", 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.1160000000000001e-06 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.980 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078583+0000", 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6290000000000004e-06 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078638+0000", 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4812000000000003e-05 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:46.078645+0000", 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.686e-06 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.981 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057335+0000", 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.97868998600000001 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057478+0000", 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014334499999999999 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.982 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.057659+0000", 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00018098100000000001 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25901 3.19 3:9f8aebfb:::rbd_data.11064477c615.0000000000000009:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 34.812501373000003, 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62440548799999995, 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:21.983 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25901 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075745+0000", 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075746+0000", 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.4099999999999998e-07 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075750+0000", 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4599999999999996e-06 2024-09-11T09:31:21.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075766+0000", 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.5579000000000001e-05 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075773+0000", 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.6839999999999996e-06 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.986 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075846+0000", 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.2606000000000002e-05 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.075864+0000", 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.766e-05 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.695967+0000", 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62010379199999999 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.987 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696129+0000", 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00016192800000000001 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.696251+0000", 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000121199 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698192+0000", 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0019412120000000001 2024-09-11T09:31:21.988 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700042+0000", 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001850317 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700107+0000", 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4455999999999994e-05 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:21.989 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700150+0000", 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3853999999999997e-05 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:21.990 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:21.991 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:21.992+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:21.992 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T09:31:22.092 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:22.093+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:22.140 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:22.140 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:22.140 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:22.140 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:22.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-11T09:31:22.193 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:22.194+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:22.255 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:22.256 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:22.256 INFO:teuthology.orchestra.run.smithi017.stdout: "complaint_time": 30, 2024-09-11T09:31:22.256 INFO:teuthology.orchestra.run.smithi017.stdout: "num_blocked_ops": 0 2024-09-11T09:31:22.256 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:22.264 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-11T09:31:22.294 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:22.296+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "size": 20, 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 600, 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [ 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8706 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2762752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181866085999999, 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026935516999999999, 2024-09-11T09:31:22.373 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8706 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.374 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190614+0000", 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.1399999999999998e-07 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190616+0000", 2024-09-11T09:31:22.375 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190623+0000", 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1970000000000001e-06 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190629+0000", 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9320000000000001e-06 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217450+0000", 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026821158000000001 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217496+0000", 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6669999999999999e-05 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217549+0000", 2024-09-11T09:31:22.377 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3034000000000001e-05 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8707 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2777088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181845366999994, 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027031756000000001, 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.378 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8707 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190635+0000", 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0900000000000002e-07 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.380 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190637+0000", 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.9700000000000002e-06 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190643+0000", 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190649+0000", 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3959999999999997e-06 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217569+0000", 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026920019 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217611+0000", 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2534000000000003e-05 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217666+0000", 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4957000000000003e-05 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.382 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8708 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2791424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181824848999995, 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027135827000000001, 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.383 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8708 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190655+0000", 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190656+0000", 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3099999999999998e-07 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190658+0000", 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2869999999999999e-06 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.385 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190664+0000", 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4930000000000003e-06 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190669+0000", 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9690000000000002e-06 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217689+0000", 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027019529 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.386 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217739+0000", 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9591e-05 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217791+0000", 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2426999999999998e-05 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8709 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2805760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.387 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181802192000006, 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027229559, 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8709 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.388 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190678+0000", 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3299999999999998e-07 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.389 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190684+0000", 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9959999999999999e-06 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190690+0000", 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3709999999999996e-06 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190696+0000", 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0690000000000001e-06 2024-09-11T09:31:22.390 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217812+0000", 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027116432999999999 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217856+0000", 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4450000000000003e-05 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.391 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217907+0000", 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0906999999999999e-05 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8710 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2830336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181776470000003, 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027330126, 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.392 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8710 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.393 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190704+0000", 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190706+0000", 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0190000000000001e-06 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.394 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190713+0000", 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6089999999999997e-06 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190717+0000", 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6909999999999996e-06 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217931+0000", 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027214228 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.395 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.217980+0000", 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8498999999999998e-05 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218034+0000", 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3721000000000003e-05 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.396 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8711 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2844672~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181756383999996, 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027422914999999999, 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8711 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.397 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.398 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190724+0000", 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190726+0000", 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0269999999999998e-06 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190732+0000", 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.9100000000000002e-06 2024-09-11T09:31:22.399 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190737+0000", 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8919999999999999e-06 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218053+0000", 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027316448 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.400 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218095+0000", 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2225000000000003e-05 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218147+0000", 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1057000000000003e-05 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.401 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8712 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2859008~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181722858000001, 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027514134999999999, 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8712 2024-09-11T09:31:22.402 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190757+0000", 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.403 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190758+0000", 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.4000000000000002e-07 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190760+0000", 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2390000000000001e-06 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190767+0000", 2024-09-11T09:31:22.404 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 7.204e-06 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190773+0000", 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.451e-06 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218171+0000", 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027398256999999999 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.405 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218218+0000", 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7537000000000002e-05 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218271+0000", 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2907e-05 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.406 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8713 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2873344~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181699887999997, 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027609320999999999, 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8713 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.407 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190780+0000", 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.408 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190781+0000", 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0399999999999996e-07 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190783+0000", 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4150000000000002e-06 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190789+0000", 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.3409999999999999e-06 2024-09-11T09:31:22.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190795+0000", 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218290+0000", 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027495622000000001 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218334+0000", 2024-09-11T09:31:22.410 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3939000000000001e-05 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218389+0000", 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5342e-05 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8714 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2883584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:22.411 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181679497999994, 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027713920999999999, 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8714 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.412 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190800+0000", 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190801+0000", 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190803+0000", 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2680000000000001e-06 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190809+0000", 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.8830000000000003e-06 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190814+0000", 2024-09-11T09:31:22.414 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0270000000000003e-06 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218415+0000", 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027600604000000001 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218462+0000", 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7219999999999999e-05 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.415 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218514+0000", 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2581999999999999e-05 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8715 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2897920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181659474, 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027806262000000002, 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.416 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8715 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.417 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190821+0000", 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.72e-07 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190823+0000", 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.018e-06 2024-09-11T09:31:22.418 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190829+0000", 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2899999999999999e-06 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190834+0000", 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8609999999999997e-06 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.419 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218533+0000", 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027699014000000001 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218576+0000", 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.3278999999999997e-05 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218627+0000", 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0427999999999999e-05 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.420 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8716 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2912256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181639863000001, 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027910032000000001, 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.421 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8716 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190840+0000", 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4200000000000002e-07 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190843+0000", 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.423 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190848+0000", 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7790000000000001e-06 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190858+0000", 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4560000000000006e-06 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218651+0000", 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027792943000000001 2024-09-11T09:31:22.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218697+0000", 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6573999999999998e-05 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218750+0000", 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2876e-05 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8717 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2926592~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181614828999997, 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0280165, 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8717 2024-09-11T09:31:22.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190865+0000", 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.427 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190866+0000", 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0999999999999999e-07 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190868+0000", 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190876+0000", 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 8.5960000000000004e-06 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190882+0000", 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.3079999999999998e-06 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218771+0000", 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027888909999999999 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218815+0000", 2024-09-11T09:31:22.429 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4081e-05 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218882+0000", 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6910999999999997e-05 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5593.0:8718 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 2940928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:22.430 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 73.181590912000004, 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028123658999999999, 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5593", 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 8718 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190889+0000", 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.5199999999999998e-07 2024-09-11T09:31:22.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190892+0000", 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2010000000000002e-06 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190898+0000", 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1580000000000002e-06 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.433 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.190902+0000", 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218908+0000", 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.028005762999999999 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.218957+0000", 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9274000000000003e-05 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:09.219013+0000", 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5411999999999997e-05 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4332 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.518199068000001, 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.026933872000000001, 2024-09-11T09:31:22.435 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4332 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854281+0000", 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.2500000000000001e-07 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.437 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854288+0000", 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.9280000000000004e-06 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854295+0000", 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4969999999999998e-06 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854300+0000", 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.2320000000000001e-06 2024-09-11T09:31:22.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879272+0000", 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024972046000000001 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879274+0000", 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.1579999999999999e-06 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881126+0000", 2024-09-11T09:31:22.439 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0018516240000000001 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881166+0000", 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9863000000000001e-05 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881215+0000", 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9098999999999999e-05 2024-09-11T09:31:22.440 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4334 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 851968~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.518171456999994, 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027023993, 2024-09-11T09:31:22.441 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4334 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.442 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854309+0000", 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.7899999999999999e-07 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854311+0000", 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.4119999999999999e-06 2024-09-11T09:31:22.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854318+0000", 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.4139999999999999e-06 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854323+0000", 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1580000000000004e-06 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.444 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879302+0000", 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024978731000000001 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879305+0000", 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.7610000000000001e-06 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881240+0000", 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001934453 2024-09-11T09:31:22.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881281+0000", 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.108e-05 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881333+0000", 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1505000000000002e-05 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4335 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 866304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.518150551999994, 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027119832999999999, 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4335 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.447 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854329+0000", 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.448 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854330+0000", 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0699999999999998e-07 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854332+0000", 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2730000000000001e-06 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854338+0000", 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.2380000000000002e-06 2024-09-11T09:31:22.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854343+0000", 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0050000000000004e-06 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879352+0000", 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025008767000000001 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:22.450 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879357+0000", 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0590000000000002e-06 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881359+0000", 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020015250000000001 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881399+0000", 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0153e-05 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.451 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881449+0000", 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0405999999999998e-05 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4337 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 880640~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.518130205999995, 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027216214999999998, 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.452 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4337 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.453 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854350+0000", 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.3700000000000001e-07 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854352+0000", 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2120000000000002e-06 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.454 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854358+0000", 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.1500000000000004e-06 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854364+0000", 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.0529999999999997e-06 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879397+0000", 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.025033715000000002 2024-09-11T09:31:22.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879402+0000", 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.6519999999999997e-06 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881476+0000", 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0020736370000000001 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.456 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881517+0000", 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.1777999999999999e-05 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881566+0000", 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.8680999999999997e-05 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4338 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 894976~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:22.457 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.517946385000002, 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027149717, 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4338 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:22.458 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854534+0000", 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.5799999999999999e-07 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.459 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854538+0000", 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.6509999999999999e-06 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854547+0000", 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.3200000000000006e-06 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854553+0000", 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.0360000000000003e-06 2024-09-11T09:31:22.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879435+0000", 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881566000000001 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879438+0000", 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.4470000000000002e-06 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881592+0000", 2024-09-11T09:31:22.461 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0021537169999999999 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881632+0000", 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 3.9910000000000002e-05 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881683+0000", 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.1511999999999997e-05 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.462 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.5721.0:4340 3.0 3:069dbf8a:::rbd_data.1106569315c1.0000000000000003:23 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88)", 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 71.517916630000002, 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.027236058, 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "started", 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.5721", 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/2823304549", 2024-09-11T09:31:22.463 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 4340 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854563+0000", 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854564+0000", 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.51e-07 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854566+0000", 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.5289999999999998e-06 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854573+0000", 2024-09-11T09:31:22.465 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 6.6189999999999996e-06 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.854579+0000", 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7259999999999999e-06 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879461+0000", 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.024881871999999999 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.466 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting_for_map not empty", 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.879463+0000", 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 2.2220000000000001e-06 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881710+0000", 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.0022475080000000001 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881750+0000", 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.0024000000000002e-05 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.467 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:10.881799+0000", 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 4.9107000000000003e-05 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "description": "osd_op(client.4358.0:25910 3.10 3:0f4297ff:::rbd_data.11064477c615.0000000000000012:head [delete] snapc 21=[] ondisk+write+known_if_redirected+supports_pool_eio e119)", 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "initiated_at": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "age": 35.294620131000002, 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.62255520900000005, 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "continuous": false, 2024-09-11T09:31:22.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type_data": { 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "client_info": { 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "client": "client.4358", 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "tid": 25910 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "events": [ 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "initiated", 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "header_read", 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077860+0000", 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "throttled", 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077861+0000", 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.545e-06 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "all_read", 2024-09-11T09:31:22.470 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077867+0000", 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 5.7309999999999998e-06 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "dispatched", 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077879+0000", 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 1.1542000000000001e-05 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.077888+0000", 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.4390000000000006e-06 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078198+0000", 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000309914 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for peered", 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.078208+0000", 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.533e-06 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "reached_pg", 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697017+0000", 2024-09-11T09:31:22.472 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.61880945499999995 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "started", 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697299+0000", 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00028221499999999999 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "waiting for subops from 5", 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.697443+0000", 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.00014368700000000001 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.473 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "op_commit", 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.698903+0000", 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001459835 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700188+0000", 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.001284795 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "commit_sent", 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700325+0000", 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 0.000137078 2024-09-11T09:31:22.474 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: "event": "done", 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: "time": "2024-09-11T09:30:47.700415+0000", 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: "duration": 9.0439999999999995e-05 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-11T09:31:22.475 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:22.477 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-11T09:31:22.477 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-11T09:31:22.477+0000 7f0926de2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32192) UID: 0 2024-09-11T09:31:22.578 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:22.578+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:22.585 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:22.585 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:22.585 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:22.585 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:22.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-11T09:31:22.679 INFO:tasks.ceph.osd.3.smithi096.stderr:2024-09-11T09:31:22.680+0000 7f9d6321d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 31685) UID: 0 2024-09-11T09:31:22.737 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:22.738 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:22.738 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:22.738 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:22.738 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:22.746 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-11T09:31:22.787 INFO:tasks.ceph.osd.4.smithi096.stderr:2024-09-11T09:31:22.788+0000 7f66163ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 31688) UID: 0 2024-09-11T09:31:22.863 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is set 2024-09-11T09:31:22.892 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:22.892+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4171.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 172.47030860300001, 2024-09-11T09:31:22.908 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36975551899999998, 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4171", 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.96:0/234940097", 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:22.909 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.436999+0000", 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.912 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437002+0000", 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.103e-06 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437009+0000", 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9689999999999997e-06 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437021+0000", 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2231e-05 2024-09-11T09:31:22.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437030+0000", 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.679e-06 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437095+0000", 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4053000000000005e-05 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:22.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.437106+0000", 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1306e-05 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805383+0000", 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.36827764299999999 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805532+0000", 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014841999999999999 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 0", 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.805656+0000", 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012419800000000001 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806336+0000", 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00067952799999999999 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806684+0000", 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00034856600000000002 2024-09-11T09:31:22.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806725+0000", 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1044000000000002e-05 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:28:30.806755+0000", 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9779000000000001e-05 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:937 3.4 3:251d12bb:::rbd_data.1106b643ff7f.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.854402875999995, 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066092815999999999, 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 937 2024-09-11T09:31:22.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052905+0000", 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.919 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052906+0000", 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5600000000000001e-07 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052908+0000", 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7970000000000001e-06 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.052998+0000", 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9421999999999999e-05 2024-09-11T09:31:22.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053004+0000", 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9449999999999998e-06 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104108+0000", 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.051104276999999997 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104173+0000", 2024-09-11T09:31:22.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5152000000000002e-05 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.104360+0000", 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00018678600000000001 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112764+0000", 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0084037609999999992 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118936+0000", 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0061727220000000003 2024-09-11T09:31:22.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118976+0000", 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9724000000000001e-05 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118998+0000", 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1874000000000001e-05 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:942 3.14 3:2cadcf2e:::rbd_data.1106b643ff7f.000000000000002c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.854205175000004, 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065934228999999997, 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 942 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053103+0000", 2024-09-11T09:31:22.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8200000000000001e-07 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053106+0000", 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6579999999999998e-06 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053112+0000", 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6470000000000003e-06 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053116+0000", 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8890000000000001e-06 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109380+0000", 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.056263138999999997 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109468+0000", 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8095000000000003e-05 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.109640+0000", 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000172041 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112971+0000", 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0033310050000000002 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.118928+0000", 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0059572649999999998 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119004+0000", 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5580000000000005e-05 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119037+0000", 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3528000000000002e-05 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:958 3.4 3:25a4a3a8:::rbd_data.1106b643ff7f.000000000000002d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.853946749000002, 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067101494999999997, 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 958 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.930 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053361+0000", 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053362+0000", 2024-09-11T09:31:22.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5100000000000001e-07 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053374+0000", 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.2116000000000001e-05 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053380+0000", 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9379999999999997e-06 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.932 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053384+0000", 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1489999999999996e-06 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.111944+0000", 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058560483000000003 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112012+0000", 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7741000000000001e-05 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112192+0000", 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017951100000000001 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120346+0000", 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0081539709999999994 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120383+0000", 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7721e-05 2024-09-11T09:31:22.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120428+0000", 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4878e-05 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120463+0000", 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4635999999999998e-05 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:959 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.853916865000002, 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065651409999999993, 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 959 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.936 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053391+0000", 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053392+0000", 2024-09-11T09:31:22.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3000000000000002e-07 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053394+0000", 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.159e-06 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053399+0000", 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0320000000000003e-06 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053403+0000", 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8149999999999999e-06 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112125+0000", 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.058722312999999998 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.112185+0000", 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9896999999999999e-05 2024-09-11T09:31:22.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119043+0000", 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.006857864 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:15160 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [watch reconnect cookie 140373489998704 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.853383712999999, 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.068254679999999998, 2024-09-11T09:31:22.940 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 15160 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053924+0000", 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053925+0000", 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1800000000000001e-07 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.942 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053927+0000", 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1430000000000001e-06 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053933+0000", 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3890000000000002e-06 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.053938+0000", 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8879999999999996e-06 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116485+0000", 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062546319000000003 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.116979+0000", 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049422500000000005 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122179+0000", 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0052002979999999999 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:968 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:22.945 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.852506263999999, 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064448565999999999, 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 968 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:22.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054802+0000", 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054803+0000", 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1299999999999998e-07 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054810+0000", 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2320000000000004e-06 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054821+0000", 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1231e-05 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054828+0000", 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9399999999999996e-06 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119069+0000", 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064240484 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119137+0000", 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8146999999999997e-05 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119250+0000", 2024-09-11T09:31:22.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113619 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:969 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.852467297999993, 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064702487000000003, 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.950 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 969 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054841+0000", 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7599999999999997e-07 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054845+0000", 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3689999999999998e-06 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.952 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054852+0000", 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.878e-06 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054857+0000", 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.091e-06 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119298+0000", 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064441066000000005 2024-09-11T09:31:22.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119389+0000", 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1379999999999996e-05 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.119543+0000", 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000154127 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:970 3.14 3:2febafbd:::rbd_data.1106b643ff7f.000000000000008a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.852423822000006, 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.067390078000000006, 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 970 2024-09-11T09:31:22.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054884+0000", 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054885+0000", 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9200000000000001e-07 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054888+0000", 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1109999999999999e-06 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054895+0000", 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2069999999999998e-06 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.054901+0000", 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4770000000000002e-06 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120039+0000", 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065138023000000003 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120130+0000", 2024-09-11T09:31:22.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1399999999999999e-05 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122274+0000", 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0021443679999999998 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:974 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537042288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.852299204999994, 2024-09-11T09:31:22.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065621703000000003, 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 974 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055009+0000", 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055010+0000", 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.92e-07 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055014+0000", 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3599999999999998e-06 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055023+0000", 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9279999999999999e-06 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055030+0000", 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0720000000000002e-06 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.962 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120455+0000", 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065425205 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120525+0000", 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.9714000000000004e-05 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120631+0000", 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105732 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.963 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:977 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537053968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.851733734999996, 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065274128000000001, 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.964 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 977 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055574+0000", 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055575+0000", 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6000000000000003e-07 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055579+0000", 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2889999999999999e-06 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055587+0000", 2024-09-11T09:31:22.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8599999999999993e-06 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055593+0000", 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2099999999999998e-06 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120660+0000", 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065066869999999999 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120744+0000", 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3616999999999996e-05 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120848+0000", 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104522 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.968 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:980 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537089136] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.851703099000005, 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065446118999999997, 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 980 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055605+0000", 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055606+0000", 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5300000000000004e-07 2024-09-11T09:31:22.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055609+0000", 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.1970000000000001e-06 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055616+0000", 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.939e-06 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055621+0000", 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3449999999999996e-06 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120882+0000", 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065261332000000005 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.120948+0000", 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5993000000000003e-05 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121051+0000", 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010266 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1014 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537112864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.849566667000005, 2024-09-11T09:31:22.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064345894000000001, 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1014 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.974 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057741+0000", 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057742+0000", 2024-09-11T09:31:22.975 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7500000000000001e-07 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057744+0000", 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4949999999999998e-06 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057752+0000", 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6860000000000006e-06 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.976 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.057756+0000", 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3710000000000002e-06 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121819+0000", 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064062419999999995 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121989+0000", 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017016700000000001 2024-09-11T09:31:22.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.122087+0000", 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8380000000000003e-05 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1025 3.4 3:25c54a99:::rbd_data.1106b643ff7f.0000000000000094:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.848046178999994, 2024-09-11T09:31:22.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.066181905999999999, 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1025 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059262+0000", 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059263+0000", 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7599999999999996e-07 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059267+0000", 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0729999999999998e-06 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059277+0000", 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8020000000000003e-06 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.059283+0000", 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8310000000000003e-06 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121799+0000", 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062515351999999996 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.121906+0000", 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000107598 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.125444+0000", 2024-09-11T09:31:22.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0035374740000000001 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1079 3.14 3:2caf3fe2:::rbd_data.1106b643ff7f.0000000000000099:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.843972350000001, 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065741193000000003, 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:22.983 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1079 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:22.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063336+0000", 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5700000000000002e-07 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063339+0000", 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.6410000000000002e-06 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063345+0000", 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3080000000000004e-06 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063350+0000", 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8550000000000001e-06 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126138+0000", 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.062787832000000002 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.126214+0000", 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.6310999999999995e-05 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129077+0000", 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0028626889999999999 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1087 3.14 3:284d155c:::rbd_data.1106b643ff7f.000000000000003a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.843858556000001, 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065713648999999999, 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1087 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063450+0000", 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0200000000000003e-07 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063453+0000", 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2090000000000001e-06 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063459+0000", 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1539999999999999e-06 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063464+0000", 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8489999999999997e-06 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127047+0000", 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063582746999999995 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127097+0000", 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9613000000000001e-05 2024-09-11T09:31:22.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 4", 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127208+0000", 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111082 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128551+0000", 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0013438670000000001 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129116+0000", 2024-09-11T09:31:22.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00056423900000000002 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129143+0000", 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7572e-05 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.129163+0000", 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9915000000000002e-05 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:22.993 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1094 3.4 3:22ee6a01:::rbd_data.1106b643ff7f.0000000000000033:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.843773873999993, 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.065101646999999999, 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1094 2024-09-11T09:31:22.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063534+0000", 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063535+0000", 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8299999999999997e-07 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063537+0000", 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7180000000000001e-06 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063544+0000", 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3990000000000001e-06 2024-09-11T09:31:22.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063549+0000", 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.186e-06 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127457+0000", 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.063908097999999997 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127526+0000", 2024-09-11T09:31:22.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8526000000000001e-05 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 1", 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127627+0000", 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000101697 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128540+0000", 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00091245699999999998 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128556+0000", 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6523000000000001e-05 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128599+0000", 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3183000000000003e-05 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128636+0000", 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6377000000000001e-05 2024-09-11T09:31:22.999 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1107 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537382192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.843727883, 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064457974000000001, 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.000 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1107 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063580+0000", 2024-09-11T09:31:23.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063581+0000", 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7700000000000005e-07 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063588+0000", 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0439999999999998e-06 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.002 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063594+0000", 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063599+0000", 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7670000000000003e-06 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127803+0000", 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064204396999999996 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.127877+0000", 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3376999999999999e-05 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128038+0000", 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00016142399999999999 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1110 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537346544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.843356584999995, 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064556769999999999, 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1110 2024-09-11T09:31:23.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063951+0000", 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063952+0000", 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7000000000000004e-07 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063957+0000", 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9620000000000001e-06 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063965+0000", 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3280000000000006e-06 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063971+0000", 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6779999999999997e-06 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128070+0000", 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064098663 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128324+0000", 2024-09-11T09:31:23.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025423299999999998 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128508+0000", 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000184236 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1113 3.14 3:2a70f94d:::rbd_header.1106b643ff7f:head [notify cookie 139945537416336] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 95.843214635999999, 2024-09-11T09:31:23.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064747263999999999, 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1113 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:23.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064093+0000", 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064094+0000", 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8700000000000005e-07 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064101+0000", 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.472e-06 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064120+0000", 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8910999999999999e-05 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.064130+0000", 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0898e-05 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128630+0000", 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.064499762000000002 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128736+0000", 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000105986 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.128841+0000", 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000104548 2024-09-11T09:31:23.013 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.014 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.014 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:23.016 INFO:teuthology.orchestra.run.smithi107.stdout:All operations completed A-OK! 2024-09-11T09:31:23.016 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-11T09:31:23.017 INFO:tasks.ceph.osd.5.smithi096.stderr:2024-09-11T09:31:23.016+0000 7fb1a43e8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 31689) UID: 0 2024-09-11T09:31:23.017 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:23.017 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-11T09:31:23.026 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-11T09:31:23.080 INFO:tasks.thrashosds:joining thrashosds 2024-09-11T09:31:23.196 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:23.196 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:23.196 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:23.205 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-11T09:31:23.317 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:23.317 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:23.317 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:23.318 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:23.318 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:23.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-11T09:31:23.445 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:23.445 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11197 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1935360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105261835999997, 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046852751999999998, 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11197 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.446 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.447 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339416+0000", 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1699999999999998e-07 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339419+0000", 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3989999999999998e-06 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339426+0000", 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.2370000000000003e-06 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.448 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339431+0000", 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5380000000000002e-06 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386095+0000", 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046663917999999999 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.449 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386173+0000", 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7696999999999998e-05 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386269+0000", 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5445999999999994e-05 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.450 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11198 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1949696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105238616999998, 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047046145999999997, 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11198 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339439+0000", 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.72e-07 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339441+0000", 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9640000000000002e-06 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339448+0000", 2024-09-11T09:31:23.453 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.477e-06 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339459+0000", 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1068e-05 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386309+0000", 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.046849654999999997 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.454 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386389+0000", 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0136999999999995e-05 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386485+0000", 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6472999999999996e-05 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11199 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1964032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.455 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105211456000006, 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047218597000000001, 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11199 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.456 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339466+0000", 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339467+0000", 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.46e-07 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.457 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339469+0000", 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339475+0000", 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.3579999999999999e-06 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339480+0000", 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3000000000000001e-06 2024-09-11T09:31:23.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386516+0000", 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047035441999999997 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386595+0000", 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.9259999999999997e-05 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386685+0000", 2024-09-11T09:31:23.459 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.9690000000000004e-05 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11200 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1974272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105190930999996, 2024-09-11T09:31:23.460 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047443780999999997, 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11200 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.461 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339487+0000", 2024-09-11T09:31:23.465 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.63e-07 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339489+0000", 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0470000000000001e-06 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339501+0000", 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1674e-05 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.466 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339506+0000", 2024-09-11T09:31:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2050000000000001e-06 2024-09-11T09:31:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.467 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386731+0000", 2024-09-11T09:31:23.468 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047224598 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386819+0000", 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8641999999999996e-05 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386930+0000", 2024-09-11T09:31:23.469 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000111152 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11201 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 1988608~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105164497999994, 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047636039999999998, 2024-09-11T09:31:23.470 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11201 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.471 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339513+0000", 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.472 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339514+0000", 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2300000000000001e-07 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339516+0000", 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.2840000000000001e-06 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339522+0000", 2024-09-11T09:31:23.473 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1099999999999999e-06 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339527+0000", 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3120000000000001e-06 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.386970+0000", 2024-09-11T09:31:23.474 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047442531000000003 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387053+0000", 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3128999999999996e-05 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387149+0000", 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6051000000000006e-05 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.475 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11202 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2002944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105136345999995, 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047829478000000002, 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11202 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:23.477 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339541+0000", 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339542+0000", 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3600000000000004e-07 2024-09-11T09:31:23.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339544+0000", 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.3690000000000001e-06 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339553+0000", 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.2250000000000006e-06 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339559+0000", 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2789999999999998e-06 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387193+0000", 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047634217 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.480 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387277+0000", 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.3788000000000005e-05 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387371+0000", 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4063999999999999e-05 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.481 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11203 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2017280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105105965999996, 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048004245000000001, 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.482 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11203 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:23.483 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339572+0000", 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.68e-07 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339575+0000", 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.4279999999999999e-06 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.484 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339582+0000", 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.3019999999999997e-06 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339587+0000", 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2900000000000002e-06 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.485 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387411+0000", 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047823800999999999 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387494+0000", 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2979999999999995e-05 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387576+0000", 2024-09-11T09:31:23.486 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.1876e-05 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11204 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2041856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105082891999999, 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048220979999999997, 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.487 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11204 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.488 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339595+0000", 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1100000000000001e-07 2024-09-11T09:31:23.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339597+0000", 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9420000000000002e-06 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339609+0000", 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1581e-05 2024-09-11T09:31:23.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339614+0000", 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2270000000000001e-06 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387604+0000", 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.047989733999999999 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.491 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387682+0000", 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8072999999999996e-05 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387816+0000", 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000134012 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11205 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2056192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105053530999996, 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048412455, 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.493 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11205 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339624+0000", 2024-09-11T09:31:23.494 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339625+0000", 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8999999999999997e-07 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339627+0000", 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.495 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339633+0000", 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7989999999999996e-06 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339639+0000", 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2959999999999998e-06 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.496 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387856+0000", 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048217031 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.387940+0000", 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.4295000000000005e-05 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388037+0000", 2024-09-11T09:31:23.497 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.6481e-05 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11206 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2070528~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105032304999995, 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048603944000000003, 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11206 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.499 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339645+0000", 2024-09-11T09:31:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.500 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339646+0000", 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339648+0000", 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.886e-06 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339654+0000", 2024-09-11T09:31:23.501 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0669999999999997e-06 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339658+0000", 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.6450000000000004e-06 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388067+0000", 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048409107999999999 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.502 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388166+0000", 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.9099999999999996e-05 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388249+0000", 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.2750999999999994e-05 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.503 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11207 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2084864~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.105012012000003, 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048730374, 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.504 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11207 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.505 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339666+0000", 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1300000000000001e-07 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339668+0000", 2024-09-11T09:31:23.506 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.906e-06 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339674+0000", 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4880000000000004e-06 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.507 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339679+0000", 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8749999999999999e-06 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388277+0000", 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048598067000000002 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.508 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388321+0000", 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.3334999999999998e-05 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388396+0000", 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5290000000000006e-05 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.509 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11208 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2109440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104985964999997, 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048883778000000003, 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11208 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339692+0000", 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9599999999999999e-07 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339694+0000", 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1260000000000001e-06 2024-09-11T09:31:23.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339701+0000", 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7719999999999997e-06 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339706+0000", 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0100000000000003e-06 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388426+0000", 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048719966000000003 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388504+0000", 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8412999999999996e-05 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.514 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388575+0000", 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.0994999999999994e-05 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11209 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2123776~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104964383999999, 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.04896114, 2024-09-11T09:31:23.515 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11209 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:23.516 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339713+0000", 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339714+0000", 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8400000000000004e-07 2024-09-11T09:31:23.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339716+0000", 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.1009999999999999e-06 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339723+0000", 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6390000000000003e-06 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.518 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339728+0000", 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.13e-06 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388591+0000", 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048863761999999998 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.519 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388629+0000", 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7772999999999999e-05 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388674+0000", 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5151000000000002e-05 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.520 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11210 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2138112~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104943581000001, 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049095044999999997, 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.521 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11210 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.522 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339734+0000", 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7099999999999997e-07 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.523 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339736+0000", 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.8470000000000001e-06 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339742+0000", 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1709999999999999e-06 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339748+0000", 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1279999999999999e-06 2024-09-11T09:31:23.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388689+0000", 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.048941363000000002 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388748+0000", 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.9453999999999997e-05 2024-09-11T09:31:23.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388829+0000", 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0710999999999993e-05 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11211 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2152448~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.526 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104924170999993, 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049272528000000003, 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11211 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.527 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339753+0000", 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339754+0000", 2024-09-11T09:31:23.528 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1099999999999996e-07 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339764+0000", 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.893e-06 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339771+0000", 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7619999999999998e-06 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.529 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339776+0000", 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1789999999999999e-06 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388847+0000", 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049071122000000002 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.388925+0000", 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8573999999999997e-05 2024-09-11T09:31:23.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389026+0000", 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000100487 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11212 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2162688~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:23.531 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104887203999994, 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049472870000000002, 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11212 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.532 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339790+0000", 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.533 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339791+0000", 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9699999999999996e-07 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339793+0000", 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.311e-06 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.534 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339803+0000", 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.5810000000000003e-06 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339808+0000", 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4430000000000002e-06 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.535 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389068+0000", 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049260072000000002 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389154+0000", 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.6033999999999997e-05 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389263+0000", 2024-09-11T09:31:23.536 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00010893200000000001 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11213 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2177024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104854802000006, 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049650605, 2024-09-11T09:31:23.537 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11213 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.538 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.539 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339823+0000", 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2499999999999995e-07 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339826+0000", 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.289e-06 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339833+0000", 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1640000000000004e-06 2024-09-11T09:31:23.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339838+0000", 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3279999999999996e-06 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389293+0000", 2024-09-11T09:31:23.541 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049455358999999997 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389382+0000", 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8558000000000002e-05 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389473+0000", 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1382e-05 2024-09-11T09:31:23.542 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11214 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2191360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104832634999994, 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049810210000000001, 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.543 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11214 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:23.544 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339845+0000", 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.01e-07 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.545 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339847+0000", 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.995e-06 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339854+0000", 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.6329999999999999e-06 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339859+0000", 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8300000000000003e-06 2024-09-11T09:31:23.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389518+0000", 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049658718999999997 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389598+0000", 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.0247999999999995e-05 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389655+0000", 2024-09-11T09:31:23.547 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7383999999999998e-05 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11215 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2205696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104807655000002, 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050014696999999997, 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11215 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.549 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339870+0000", 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8599999999999998e-07 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.551 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339872+0000", 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0549999999999998e-06 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339879+0000", 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.2609999999999999e-06 2024-09-11T09:31:23.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339884+0000", 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8779999999999997e-06 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.553 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389698+0000", 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.049814002000000003 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389786+0000", 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8046000000000004e-05 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.554 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389885+0000", 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.8968999999999994e-05 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5593.0:11216 3.9 3:93df5701:::rbd_data.1106569315c1.0000000000000001:23 [read 2220032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 74.104787275999996, 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050126667999999999, 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5593", 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4132951081", 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 11216 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.557 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339890+0000", 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339891+0000", 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7599999999999998e-07 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339893+0000", 2024-09-11T09:31:23.559 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9369999999999998e-06 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339900+0000", 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5039999999999996e-06 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.560 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.339905+0000", 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.904e-06 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389912+0000", 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.050007107000000002 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.561 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.389963+0000", 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.1286999999999999e-05 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.562 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:30:09.390017+0000", 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3553e-05 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.563 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:23.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-11T09:31:23.671 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:23.671 INFO:teuthology.orchestra.run.smithi096.stdout: "num_ops": 0 2024-09-11T09:31:23.671 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:23.680 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-11T09:31:23.788 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [], 2024-09-11T09:31:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "complaint_time": 30, 2024-09-11T09:31:23.788 INFO:teuthology.orchestra.run.smithi096.stdout: "num_blocked_ops": 0 2024-09-11T09:31:23.789 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:23.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "size": 20, 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 600, 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "ops": [ 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4844 3.1c 3:3edddbdc:::rbd_data.1106cda8c42d.0000000000000001:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92935453499999, 2024-09-11T09:31:23.913 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033893157, 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4844 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983121+0000", 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3300000000000002e-07 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.915 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983123+0000", 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0619999999999999e-06 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983127+0000", 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7069999999999998e-06 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983131+0000", 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4159999999999997e-06 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.916 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016918+0000", 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033786251000000003 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.016959+0000", 2024-09-11T09:31:23.917 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1517999999999998e-05 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017014+0000", 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4669999999999997e-05 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.918 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4848 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 282624~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.929308491, 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033968901000000003, 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.919 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4848 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.920 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983167+0000", 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4999999999999998e-07 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.921 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983169+0000", 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.6789999999999999e-06 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983172+0000", 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4879999999999999e-06 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983175+0000", 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7590000000000002e-06 2024-09-11T09:31:23.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017038+0000", 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033863191000000001 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017082+0000", 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4199999999999997e-05 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.923 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017136+0000", 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.3233999999999999e-05 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4853 3.1c 3:3bda783f:::rbd_data.1106cda8c42d.0000000000000008:a [read 284672~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92929682100001, 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034076710000000003, 2024-09-11T09:31:23.924 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4853 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983178+0000", 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983868+0000", 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00068943300000000004 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.926 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983872+0000", 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7639999999999999e-06 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983881+0000", 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.149e-06 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.983887+0000", 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.1299999999999998e-06 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.927 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017160+0000", 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033272726000000002 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.928 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017201+0000", 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1504000000000001e-05 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.017255+0000", 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4004e-05 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.929 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4943 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 704512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.930 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92797285099999, 2024-09-11T09:31:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033884324, 2024-09-11T09:31:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4943 2024-09-11T09:31:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.932 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.933 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984502+0000", 2024-09-11T09:31:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.934 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984503+0000", 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6100000000000002e-07 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984504+0000", 2024-09-11T09:31:23.935 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1519999999999999e-06 2024-09-11T09:31:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984507+0000", 2024-09-11T09:31:23.936 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5949999999999999e-06 2024-09-11T09:31:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984511+0000", 2024-09-11T09:31:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0330000000000002e-06 2024-09-11T09:31:23.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018307+0000", 2024-09-11T09:31:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033795902000000003 2024-09-11T09:31:23.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018349+0000", 2024-09-11T09:31:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1418999999999998e-05 2024-09-11T09:31:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018387+0000", 2024-09-11T09:31:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7861999999999998e-05 2024-09-11T09:31:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4946 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 716800~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:23.941 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92795081400001, 2024-09-11T09:31:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033966155999999997, 2024-09-11T09:31:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.942 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4946 2024-09-11T09:31:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.943 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.944 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984524+0000", 2024-09-11T09:31:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.945 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984525+0000", 2024-09-11T09:31:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8599999999999999e-07 2024-09-11T09:31:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.946 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984527+0000", 2024-09-11T09:31:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.029e-06 2024-09-11T09:31:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.947 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984532+0000", 2024-09-11T09:31:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4990000000000002e-06 2024-09-11T09:31:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.948 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984538+0000", 2024-09-11T09:31:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.947e-06 2024-09-11T09:31:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.949 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018411+0000", 2024-09-11T09:31:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033872344999999998 2024-09-11T09:31:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018452+0000", 2024-09-11T09:31:23.950 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1819000000000001e-05 2024-09-11T09:31:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018490+0000", 2024-09-11T09:31:23.951 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8130999999999998e-05 2024-09-11T09:31:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.952 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4949 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 464896~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92791522900001, 2024-09-11T09:31:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034032791, 2024-09-11T09:31:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4949 2024-09-11T09:31:23.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:23.955 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984560+0000", 2024-09-11T09:31:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5499999999999998e-07 2024-09-11T09:31:23.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984566+0000", 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.8239999999999996e-06 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.958 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984572+0000", 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.0580000000000003e-06 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984577+0000", 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8509999999999998e-06 2024-09-11T09:31:23.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018515+0000", 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033938109000000001 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.960 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018554+0000", 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8933000000000001e-05 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018593+0000", 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8560999999999999e-05 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.961 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4955 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 694272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92785488600001, 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034084831000000003, 2024-09-11T09:31:23.962 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4955 2024-09-11T09:31:23.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984620+0000", 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.964 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984621+0000", 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4199999999999996e-07 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984623+0000", 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0710000000000002e-06 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984632+0000", 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.8599999999999999e-06 2024-09-11T09:31:23.965 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984637+0000", 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0799999999999996e-06 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018617+0000", 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033980574999999999 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.966 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018660+0000", 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.2985999999999998e-05 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018705+0000", 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4716999999999999e-05 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4960 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 477184~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.967 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.927806202, 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034140344000000003, 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4960 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.968 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.969 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984669+0000", 2024-09-11T09:31:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0599999999999999e-07 2024-09-11T09:31:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.970 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984670+0000", 2024-09-11T09:31:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.186e-06 2024-09-11T09:31:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.971 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984673+0000", 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.0730000000000001e-06 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984677+0000", 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.411e-06 2024-09-11T09:31:23.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018730+0000", 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034052772000000002 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.973 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018770+0000", 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0389999999999998e-05 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018809+0000", 2024-09-11T09:31:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9305999999999999e-05 2024-09-11T09:31:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.975 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4964 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 706560~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92774973799999, 2024-09-11T09:31:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034188258999999999, 2024-09-11T09:31:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4964 2024-09-11T09:31:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.977 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984725+0000", 2024-09-11T09:31:23.978 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984726+0000", 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8500000000000002e-07 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984727+0000", 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.3379999999999999e-06 2024-09-11T09:31:23.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984733+0000", 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.6019999999999999e-06 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984738+0000", 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2390000000000002e-06 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.980 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018833+0000", 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034094615000000002 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018878+0000", 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5074999999999998e-05 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018914+0000", 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6004999999999998e-05 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.981 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4965 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 489472~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92773018299999, 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034269994999999998, 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.982 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4965 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.983 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984745+0000", 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.5900000000000003e-07 2024-09-11T09:31:23.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984747+0000", 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0590000000000001e-06 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984754+0000", 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8589999999999999e-06 2024-09-11T09:31:23.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984759+0000", 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.4989999999999996e-06 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018937+0000", 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034178192000000003 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.986 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.018977+0000", 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0522999999999998e-05 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019015+0000", 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7503999999999999e-05 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.987 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4968 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 718848~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92768242, 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034324027999999999, 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4968 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.988 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.989 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984793+0000", 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.9999999999999998e-07 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984796+0000", 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.9739999999999998e-06 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984803+0000", 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8329999999999996e-06 2024-09-11T09:31:23.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984808+0000", 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.4439999999999999e-06 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019039+0000", 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034230160000000003 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.991 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019079+0000", 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0810999999999999e-05 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019117+0000", 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7305999999999998e-05 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.992 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4976 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 466944~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92756868199999, 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034309602000000002, 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4976 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.993 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984906+0000", 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.994 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984907+0000", 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8700000000000001e-07 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984909+0000", 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.063e-06 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984915+0000", 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.7710000000000003e-06 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.995 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984919+0000", 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5530000000000003e-06 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019140+0000", 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034220933000000002 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:23.996 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019179+0000", 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8692999999999997e-05 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019216+0000", 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7202000000000001e-05 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4979 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 479232~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:23.997 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.927549938, 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034392319999999997, 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4979 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:23.998 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984925+0000", 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984926+0000", 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.3999999999999997e-07 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:23.999 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984928+0000", 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.0559999999999999e-06 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984934+0000", 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.4450000000000001e-06 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984939+0000", 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.0470000000000001e-06 2024-09-11T09:31:24.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019239+0000", 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034300118999999997 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019280+0000", 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.0923000000000001e-05 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.001 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019318+0000", 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7389999999999999e-05 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4982 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 696320~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92751094900001, 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034451900000000001, 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4982 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.003 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984964+0000", 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984965+0000", 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.2800000000000003e-07 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.004 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984966+0000", 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.389e-06 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984969+0000", 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.4199999999999999e-06 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984973+0000", 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.6500000000000002e-06 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.005 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019340+0000", 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034367215999999999 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019379+0000", 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8818e-05 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019416+0000", 2024-09-11T09:31:24.006 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.7079000000000002e-05 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4756.0:4987 3.1c 3:3e72ae53:::rbd_data.11069296af76.0000000000000001:b [read 708608~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 132.92748784200001, 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.034532052000000001, 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:24.007 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4756", 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3709900157", 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 4987 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.008 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.984987+0000", 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986866+0000", 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0018787910000000001 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986871+0000", 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5510000000000001e-06 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.009 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986881+0000", 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.0173999999999999e-05 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:10.986888+0000", 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7089999999999996e-06 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019440+0000", 2024-09-11T09:31:24.010 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.032552605999999998 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019481+0000", 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1059000000000001e-05 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:11.019519+0000", 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.8161999999999998e-05 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.011 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5017.0:2591 3.f 3:f6094797:::rbd_object_map.1106623838a6:head [read 0~39] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 114.21941135100001, 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.003486659, 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5017", 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/3253308137", 2024-09-11T09:31:24.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 2591 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693064+0000", 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.013 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693065+0000", 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.347e-06 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693070+0000", 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.7029999999999997e-06 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.014 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693081+0000", 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1437000000000001e-05 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693090+0000", 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.7110000000000001e-06 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693841+0000", 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00075051399999999998 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.015 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:29.693852+0000", 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1432e-05 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696271+0000", 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.002419379 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.016 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696394+0000", 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000122955 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:30.696550+0000", 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000156181 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1013 3.16 3:680ed9d0:::rbd_data.1106b643ff7f.0000000000000076:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.857012021000003, 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.035422311999999997, 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1013 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055463+0000", 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.019 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055464+0000", 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 8.16e-07 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055469+0000", 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8659999999999997e-06 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055480+0000", 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.1054e-05 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.020 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055488+0000", 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.7260000000000002e-06 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055540+0000", 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.2763999999999998e-05 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.021 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055608+0000", 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.8128999999999998e-05 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.055786+0000", 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017718599999999999 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061705+0000", 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.005919255 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.022 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090795+0000", 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.029090436000000001 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090843+0000", 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.8109999999999998e-05 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.090885+0000", 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.197e-05 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.023 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1080 3.5 3:a003c985:::rbd_data.1106b643ff7f.000000000000009a:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.851760944000006, 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036697487000000001, 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "started", 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:24.024 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1080 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060714+0000", 2024-09-11T09:31:24.025 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060715+0000", 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.5099999999999999e-07 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060719+0000", 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.1659999999999996e-06 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.026 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060727+0000", 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.627e-06 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060733+0000", 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.5459999999999997e-06 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060780+0000", 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.689e-05 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060894+0000", 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.000113823 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097412+0000", 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036517684000000002 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.028 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.5075.0:1089 3.5 3:a4b3d3c7:::rbd_data.1106b643ff7f.0000000000000078:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 96.851500573999999, 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.036695817999999998, 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.5075", 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/4240288714", 2024-09-11T09:31:24.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 1089 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060975+0000", 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.1399999999999996e-07 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060980+0000", 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 4.5770000000000004e-06 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060989+0000", 2024-09-11T09:31:24.031 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.1179999999999998e-06 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.060996+0000", 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 6.7150000000000001e-06 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061487+0000", 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00049126599999999997 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.032 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061565+0000", 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8149e-05 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.061740+0000", 2024-09-11T09:31:24.033 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00017505099999999999 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.063805+0000", 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0020646879999999999 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097520+0000", 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.033715632000000002 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:24.034 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097576+0000", 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 5.5680000000000002e-05 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:29:47.097670+0000", 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 9.4228000000000007e-05 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.035 INFO:teuthology.orchestra.run.smithi096.stdout: "description": "osd_op(client.4358.0:29202 3.a 3:517570f5:::rbd_data.1106551ea4b0.0000000000000002:head [delete] snapc e=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "initiated_at": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 15.034161745, 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92978124200000001, 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "continuous": false, 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type_data": { 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_info": { 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client": "client.4358", 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "client_addr": "172.21.15.107:0/1158645234", 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "tid": 29202 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.036 INFO:teuthology.orchestra.run.smithi096.stdout: "events": [ 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "initiated", 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "header_read", 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878313+0000", 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "throttled", 2024-09-11T09:31:24.037 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878315+0000", 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.5999999999999999e-06 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "all_read", 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878322+0000", 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.216e-06 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "dispatched", 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878341+0000", 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.9160999999999998e-05 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "queued_for_pg", 2024-09-11T09:31:24.038 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878369+0000", 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 2.7441999999999999e-05 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878447+0000", 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 7.8467999999999995e-05 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for peered", 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:08.878462+0000", 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 1.4299e-05 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "reached_pg", 2024-09-11T09:31:24.039 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804505+0000", 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.92604313900000002 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "started", 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804629+0000", 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00012435199999999999 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "waiting for subops from 3", 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.804779+0000", 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00014989300000000001 2024-09-11T09:31:24.040 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "op_commit", 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.807759+0000", 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.0029804020000000001 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "sub_op_commit_rec", 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808015+0000", 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 0.00025595300000000001 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "commit_sent", 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808055+0000", 2024-09-11T09:31:24.041 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9987000000000002e-05 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "event": "done", 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "time": "2024-09-11T09:31:09.808095+0000", 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: "duration": 3.9329999999999998e-05 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-11T09:31:24.042 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-11T09:31:24.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-11T09:31:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:noscrub is unset 2024-09-11T09:31:25.873 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-11T09:31:25.873 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-11T09:31:25.873 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T09:31:25.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T09:31:25.876 INFO:tasks.ceph.osd.0:Started 2024-09-11T09:31:25.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:31:25.892 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:no valid command found; 10 closest matches: 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:0 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:1 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:abort 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:assert 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:config diff 2024-09-11T09:31:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:config diff get 2024-09-11T09:31:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:config get 2024-09-11T09:31:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:config help [] 2024-09-11T09:31:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:config set ... 2024-09-11T09:31:26.000 INFO:teuthology.orchestra.run.smithi017.stderr:admin_socket: invalid command 2024-09-11T09:31:26.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T09:31:26.002 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-11T09:31:26.807 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:31:26.807+0000 7f03696d6640 -1 Falling back to public interface 2024-09-11T09:31:26.879 INFO:teuthology.orchestra.run.smithi017.stderr:nodeep-scrub is unset 2024-09-11T09:31:30.400 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:31:30.400+0000 7f03696d6640 -1 osd.0 89 log_to_monitors true 2024-09-11T09:31:30.928 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-11T09:31:30.928+0000 7f035d485640 -1 osd.0 129 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T09:31:31.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T09:31:31.117 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:31.118 INFO:teuthology.orchestra.run.smithi017.stdout: "ops": [], 2024-09-11T09:31:31.118 INFO:teuthology.orchestra.run.smithi017.stdout: "num_ops": 0 2024-09-11T09:31:31.118 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:31.126 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-11T09:31:31.127 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-11T09:31:31.127 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-11T09:31:31.423 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-09-11T09:31:31.423+0000 7fef02752640 -1 mon.a@0(leader).osd e160 definitely_dead 0 2024-09-11T09:31:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:marked in osd.3. 2024-09-11T09:31:31.919 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-11T09:31:31.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-11T09:31:32.038 INFO:teuthology.orchestra.run.smithi017.stdout:mon.a: {} 2024-09-11T09:31:32.039 INFO:teuthology.orchestra.run.smithi017.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:31:32.048 INFO:teuthology.orchestra.run.smithi017.stdout:mon.b: {} 2024-09-11T09:31:32.048 INFO:teuthology.orchestra.run.smithi017.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:31:32.057 INFO:teuthology.orchestra.run.smithi017.stdout:mon.c: {} 2024-09-11T09:31:32.057 INFO:teuthology.orchestra.run.smithi017.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-11T09:31:32.067 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-11T09:31:32.696 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.1 to 1 (10000) 2024-09-11T09:31:32.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-11T09:31:33.701 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-11T09:31:33.735 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-11T09:31:34.703 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.2 to 1 (10000) 2024-09-11T09:31:34.726 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-11T09:31:35.705 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-11T09:31:35.727 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-11T09:31:36.704 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.3 to 1 (10000) 2024-09-11T09:31:36.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-11T09:31:37.712 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-11T09:31:37.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-11T09:31:38.710 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.4 to 1 (10000) 2024-09-11T09:31:38.731 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-11T09:31:39.718 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-11T09:31:39.736 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-11T09:31:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.5 to 1 (10000) 2024-09-11T09:31:40.742 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-11T09:31:41.721 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-11T09:31:41.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-11T09:31:42.694 INFO:teuthology.orchestra.run.smithi017.stderr:reweighted osd.0 to 1 (10000) 2024-09-11T09:31:42.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-11T09:31:43.701 INFO:teuthology.orchestra.run.smithi017.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-11T09:31:43.716 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-11T09:31:43.716 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-11T09:31:43.716 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-11T09:31:43.716 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-11T09:31:43.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T09:31:43.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T09:31:44.023 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:44.024 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":173,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:31:43.695354+0000","last_up_change":"2024-09-11T09:31:31.888533+0000","last_in_change":"2024-09-11T09:31:31.888533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T09:28:28.765227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T09:28:34.774979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-11T09:28:45.573818+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":153,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":89,"up_from":161,"up_thru":172,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6801","nonce":2589762321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6803","nonce":2589762321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6807","nonce":2589762321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6805","nonce":2589762321}]},"public_addr":"172.21.15.17:6801/2589762321","cluster_addr":"172.21.15.17:6803/2589762321","heartbeat_back_addr":"172.21.15.17:6807/2589762321","heartbeat_front_addr":"172.21.15.17:6805/2589762321","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6817","nonce":361445038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6819","nonce":361445038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6823","nonce":361445038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6821","nonce":361445038}]},"public_addr":"172.21.15.17:6817/361445038","cluster_addr":"172.21.15.17:6819/361445038","heartbeat_back_addr":"172.21.15.17:6823/361445038","heartbeat_front_addr":"172.21.15.17:6821/361445038","state":["exists","up"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6809","nonce":211521567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6811","nonce":211521567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6815","nonce":211521567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6813","nonce":211521567}]},"public_addr":"172.21.15.17:6809/211521567","cluster_addr":"172.21.15.17:6811/211521567","heartbeat_back_addr":"172.21.15.17:6815/211521567","heartbeat_front_addr":"172.21.15.17:6813/211521567","state":["exists","up"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6801","nonce":2070567113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6803","nonce":2070567113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6807","nonce":2070567113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6805","nonce":2070567113}]},"public_addr":"172.21.15.96:6801/2070567113","cluster_addr":"172.21.15.96:6803/2070567113","heartbeat_back_addr":"172.21.15.96:6807/2070567113","heartbeat_front_addr":"172.21.15.96:6805/2070567113","state":["exists","up"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6817","nonce":1795828331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6819","nonce":1795828331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6823","nonce":1795828331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6821","nonce":1795828331}]},"public_addr":"172.21.15.96:6817/1795828331","cluster_addr":"172.21.15.96:6819/1795828331","heartbeat_back_addr":"172.21.15.96:6823/1795828331","heartbeat_front_addr":"172.21.15.96:6821/1795828331","state":["exists","up"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6809","nonce":3808455039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6811","nonce":3808455039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6815","nonce":3808455039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6813","nonce":3808455039}]},"public_addr":"172.21.15.96:6809/3808455039","cluster_addr":"172.21.15.96:6811/3808455039","heartbeat_back_addr":"172.21.15.96:6815/3808455039","heartbeat_front_addr":"172.21.15.96:6813/3808455039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.320025+0000","dead_epoch":90},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.922939+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.876780+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.854159+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.081538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.168719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.10","osds":[0]},{"pgid":"3.16","osds":[1,3]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":3,"length":43}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:31:44.036 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T09:31:44.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T09:31:44.329 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:44.329 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":173,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:31:43.695354+0000","last_up_change":"2024-09-11T09:31:31.888533+0000","last_in_change":"2024-09-11T09:31:31.888533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T09:28:28.765227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T09:28:34.774979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-11T09:28:45.573818+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":153,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":89,"up_from":161,"up_thru":172,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6801","nonce":2589762321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6803","nonce":2589762321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6807","nonce":2589762321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6805","nonce":2589762321}]},"public_addr":"172.21.15.17:6801/2589762321","cluster_addr":"172.21.15.17:6803/2589762321","heartbeat_back_addr":"172.21.15.17:6807/2589762321","heartbeat_front_addr":"172.21.15.17:6805/2589762321","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6817","nonce":361445038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6819","nonce":361445038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6823","nonce":361445038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6821","nonce":361445038}]},"public_addr":"172.21.15.17:6817/361445038","cluster_addr":"172.21.15.17:6819/361445038","heartbeat_back_addr":"172.21.15.17:6823/361445038","heartbeat_front_addr":"172.21.15.17:6821/361445038","state":["exists","up"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6809","nonce":211521567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6811","nonce":211521567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6815","nonce":211521567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6813","nonce":211521567}]},"public_addr":"172.21.15.17:6809/211521567","cluster_addr":"172.21.15.17:6811/211521567","heartbeat_back_addr":"172.21.15.17:6815/211521567","heartbeat_front_addr":"172.21.15.17:6813/211521567","state":["exists","up"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6801","nonce":2070567113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6803","nonce":2070567113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6807","nonce":2070567113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6805","nonce":2070567113}]},"public_addr":"172.21.15.96:6801/2070567113","cluster_addr":"172.21.15.96:6803/2070567113","heartbeat_back_addr":"172.21.15.96:6807/2070567113","heartbeat_front_addr":"172.21.15.96:6805/2070567113","state":["exists","up"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6817","nonce":1795828331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6819","nonce":1795828331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6823","nonce":1795828331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6821","nonce":1795828331}]},"public_addr":"172.21.15.96:6817/1795828331","cluster_addr":"172.21.15.96:6819/1795828331","heartbeat_back_addr":"172.21.15.96:6823/1795828331","heartbeat_front_addr":"172.21.15.96:6821/1795828331","state":["exists","up"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6809","nonce":3808455039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6811","nonce":3808455039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6815","nonce":3808455039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6813","nonce":3808455039}]},"public_addr":"172.21.15.96:6809/3808455039","cluster_addr":"172.21.15.96:6811/3808455039","heartbeat_back_addr":"172.21.15.96:6815/3808455039","heartbeat_front_addr":"172.21.15.96:6813/3808455039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.320025+0000","dead_epoch":90},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.922939+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.876780+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.854159+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.081538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.168719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.10","osds":[0]},{"pgid":"3.16","osds":[1,3]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":3,"length":43}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:31:44.342 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T09:31:44.342 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T09:31:44.342 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T09:31:44.343 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T09:31:44.343 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T09:31:44.344 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T09:31:44.500 INFO:teuthology.orchestra.run.smithi017.stdout:51539607596 2024-09-11T09:31:44.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T09:31:44.501 INFO:teuthology.orchestra.run.smithi017.stdout:51539607596 2024-09-11T09:31:44.501 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T09:31:44.546 INFO:teuthology.orchestra.run.smithi017.stdout:51539607595 2024-09-11T09:31:44.547 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T09:31:44.553 INFO:teuthology.orchestra.run.smithi017.stdout:51539607596 2024-09-11T09:31:44.554 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T09:31:44.561 INFO:teuthology.orchestra.run.smithi017.stdout:51539607595 2024-09-11T09:31:44.562 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T09:31:44.568 INFO:teuthology.orchestra.run.smithi017.stdout:691489734660 2024-09-11T09:31:44.569 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T09:31:44.891 INFO:teuthology.orchestra.run.smithi017.stdout:51539607593 2024-09-11T09:31:44.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607595 got 51539607593 for osd.3 2024-09-11T09:31:44.962 INFO:teuthology.orchestra.run.smithi017.stdout:691489734659 2024-09-11T09:31:44.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 691489734660 got 691489734659 for osd.0 2024-09-11T09:31:45.054 INFO:teuthology.orchestra.run.smithi017.stdout:51539607593 2024-09-11T09:31:45.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607596 got 51539607593 for osd.5 2024-09-11T09:31:45.073 INFO:teuthology.orchestra.run.smithi017.stdout:51539607594 2024-09-11T09:31:45.075 INFO:teuthology.orchestra.run.smithi017.stdout:51539607594 2024-09-11T09:31:45.080 INFO:teuthology.orchestra.run.smithi017.stdout:51539607593 2024-09-11T09:31:45.088 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607596 got 51539607594 for osd.1 2024-09-11T09:31:45.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607596 got 51539607594 for osd.2 2024-09-11T09:31:45.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607595 got 51539607593 for osd.4 2024-09-11T09:31:45.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T09:31:45.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T09:31:46.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T09:31:46.088 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T09:31:46.091 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T09:31:46.093 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T09:31:46.362 INFO:teuthology.orchestra.run.smithi017.stdout:51539607595 2024-09-11T09:31:46.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607595 got 51539607595 for osd.3 2024-09-11T09:31:46.381 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:46.429 INFO:teuthology.orchestra.run.smithi017.stdout:691489734660 2024-09-11T09:31:46.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 691489734660 got 691489734660 for osd.0 2024-09-11T09:31:46.443 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:46.458 INFO:teuthology.orchestra.run.smithi017.stdout:51539607596 2024-09-11T09:31:46.459 INFO:teuthology.orchestra.run.smithi017.stdout:51539607596 2024-09-11T09:31:46.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607596 got 51539607596 for osd.1 2024-09-11T09:31:46.472 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:46.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607596 got 51539607596 for osd.2 2024-09-11T09:31:46.473 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:46.550 INFO:teuthology.orchestra.run.smithi017.stdout:51539607596 2024-09-11T09:31:46.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607596 got 51539607596 for osd.5 2024-09-11T09:31:46.564 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:46.582 INFO:teuthology.orchestra.run.smithi017.stdout:51539607595 2024-09-11T09:31:46.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607595 got 51539607595 for osd.4 2024-09-11T09:31:46.594 DEBUG:teuthology.parallel:result is None 2024-09-11T09:31:46.594 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-11T09:31:46.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:46.926 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:46.927 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:46.938 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-11T09:31:46.755829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352812,"kb_used_data":12244,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878228,"statfs":{"total":579820584960,"available":579459305472,"internally_reserved":0,"allocated":12537856,"data_stored":6464916,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1485,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.060111"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7159,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701106+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:43.701106+0000","last_peered":"2024-09-11T09:31:43.701106+0000","last_clean":"2024-09-11T09:31:43.701106+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:43.701106+0000","last_undegraded":"2024-09-11T09:31:43.701106+0000","last_fullsized":"2024-09-11T09:31:43.701106+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1d","version":"155'718","reported_seq":7972,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708625+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:43.708625+0000","last_peered":"2024-09-11T09:31:43.708625+0000","last_clean":"2024-09-11T09:31:43.708625+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:43.708625+0000","last_undegraded":"2024-09-11T09:31:43.708625+0000","last_fullsized":"2024-09-11T09:31:43.708625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1c","version":"155'740","reported_seq":14240,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708709+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:43.708709+0000","last_peered":"2024-09-11T09:31:43.708709+0000","last_clean":"2024-09-11T09:31:43.708709+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:43.708709+0000","last_undegraded":"2024-09-11T09:31:43.708709+0000","last_fullsized":"2024-09-11T09:31:43.708709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.711685+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:43.711685+0000","last_peered":"2024-09-11T09:31:43.711685+0000","last_clean":"2024-09-11T09:31:43.711685+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:43.711685+0000","last_undegraded":"2024-09-11T09:31:43.711685+0000","last_fullsized":"2024-09-11T09:31:43.711685+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.19","version":"151'697","reported_seq":8418,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708084+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:43.708084+0000","last_peered":"2024-09-11T09:31:43.708084+0000","last_clean":"2024-09-11T09:31:43.708084+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:43.708084+0000","last_undegraded":"2024-09-11T09:31:43.708084+0000","last_fullsized":"2024-09-11T09:31:43.708084+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.18","version":"151'490","reported_seq":3311,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700942+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:43.700942+0000","last_peered":"2024-09-11T09:31:43.700942+0000","last_clean":"2024-09-11T09:31:43.700942+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:43.700942+0000","last_undegraded":"2024-09-11T09:31:43.700942+0000","last_fullsized":"2024-09-11T09:31:43.700942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10800,"reported_epoch":173,"state":"active+remapped","last_fresh":"2024-09-11T09:31:43.957784+0000","last_change":"2024-09-11T09:31:43.958006+0000","last_active":"2024-09-11T09:31:43.957784+0000","last_peered":"2024-09-11T09:31:43.957784+0000","last_clean":"2024-09-11T09:31:36.719409+0000","last_became_active":"2024-09-11T09:31:42.729993+0000","last_became_peered":"2024-09-11T09:31:42.729993+0000","last_unstale":"2024-09-11T09:31:43.957784+0000","last_undegraded":"2024-09-11T09:31:43.957784+0000","last_fullsized":"2024-09-11T09:31:43.957784+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[1,3],"avail_no_missing":["1","3","5"],"object_location_counts":[{"shards":"1,3","objects":12}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.15","version":"157'602","reported_seq":6288,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705415+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:43.705415+0000","last_peered":"2024-09-11T09:31:43.705415+0000","last_clean":"2024-09-11T09:31:43.705415+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:43.705415+0000","last_undegraded":"2024-09-11T09:31:43.705415+0000","last_fullsized":"2024-09-11T09:31:43.705415+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.14","version":"151'682","reported_seq":8990,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705108+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:43.705108+0000","last_peered":"2024-09-11T09:31:43.705108+0000","last_clean":"2024-09-11T09:31:43.705108+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:43.705108+0000","last_undegraded":"2024-09-11T09:31:43.705108+0000","last_fullsized":"2024-09-11T09:31:43.705108+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704976+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:43.704976+0000","last_peered":"2024-09-11T09:31:43.704976+0000","last_clean":"2024-09-11T09:31:43.704976+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:43.704976+0000","last_undegraded":"2024-09-11T09:31:43.704976+0000","last_fullsized":"2024-09-11T09:31:43.704976+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701734+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:43.701734+0000","last_peered":"2024-09-11T09:31:43.701734+0000","last_clean":"2024-09-11T09:31:43.701734+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:43.701734+0000","last_undegraded":"2024-09-11T09:31:43.701734+0000","last_fullsized":"2024-09-11T09:31:43.701734+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7482,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701333+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:43.701333+0000","last_peered":"2024-09-11T09:31:43.701333+0000","last_clean":"2024-09-11T09:31:43.701333+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:43.701333+0000","last_undegraded":"2024-09-11T09:31:43.701333+0000","last_fullsized":"2024-09-11T09:31:43.701333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705331+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:43.705331+0000","last_peered":"2024-09-11T09:31:43.705331+0000","last_clean":"2024-09-11T09:31:43.705331+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:43.705331+0000","last_undegraded":"2024-09-11T09:31:43.705331+0000","last_fullsized":"2024-09-11T09:31:43.705331+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7103,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701609+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:43.701609+0000","last_peered":"2024-09-11T09:31:43.701609+0000","last_clean":"2024-09-11T09:31:43.701609+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:43.701609+0000","last_undegraded":"2024-09-11T09:31:43.701609+0000","last_fullsized":"2024-09-11T09:31:43.701609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.4","version":"0'0","reported_seq":287,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701664+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:43.701664+0000","last_peered":"2024-09-11T09:31:43.701664+0000","last_clean":"2024-09-11T09:31:43.701664+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:43.701664+0000","last_undegraded":"2024-09-11T09:31:43.701664+0000","last_fullsized":"2024-09-11T09:31:43.701664+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5963,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708753+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:43.708753+0000","last_peered":"2024-09-11T09:31:43.708753+0000","last_clean":"2024-09-11T09:31:43.708753+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:43.708753+0000","last_undegraded":"2024-09-11T09:31:43.708753+0000","last_fullsized":"2024-09-11T09:31:43.708753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.3","version":"165'990","reported_seq":13947,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704963+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:43.704963+0000","last_peered":"2024-09-11T09:31:43.704963+0000","last_clean":"2024-09-11T09:31:43.704963+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:43.704963+0000","last_undegraded":"2024-09-11T09:31:43.704963+0000","last_fullsized":"2024-09-11T09:31:43.704963+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.2","version":"19'2","reported_seq":330,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708278+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:43.708278+0000","last_peered":"2024-09-11T09:31:43.708278+0000","last_clean":"2024-09-11T09:31:43.708278+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:43.708278+0000","last_undegraded":"2024-09-11T09:31:43.708278+0000","last_fullsized":"2024-09-11T09:31:43.708278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11644,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701669+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:43.701669+0000","last_peered":"2024-09-11T09:31:43.701669+0000","last_clean":"2024-09-11T09:31:43.701669+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:43.701669+0000","last_undegraded":"2024-09-11T09:31:43.701669+0000","last_fullsized":"2024-09-11T09:31:43.701669+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700944+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:43.700944+0000","last_peered":"2024-09-11T09:31:43.700944+0000","last_clean":"2024-09-11T09:31:43.700944+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:43.700944+0000","last_undegraded":"2024-09-11T09:31:43.700944+0000","last_fullsized":"2024-09-11T09:31:43.700944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":213,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704751+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:43.704751+0000","last_peered":"2024-09-11T09:31:43.704751+0000","last_clean":"2024-09-11T09:31:43.704751+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:43.704751+0000","last_undegraded":"2024-09-11T09:31:43.704751+0000","last_fullsized":"2024-09-11T09:31:43.704751+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":5998,"reported_epoch":172,"state":"peering","last_fresh":"2024-09-11T09:31:43.458721+0000","last_change":"2024-09-11T09:31:43.702628+0000","last_active":"2024-09-11T09:31:43.702628+0000","last_peered":"2024-09-11T09:31:43.458721+0000","last_clean":"2024-09-11T09:31:39.719720+0000","last_became_active":"2024-09-11T09:31:42.722345+0000","last_became_peered":"2024-09-11T09:31:42.722345+0000","last_unstale":"2024-09-11T09:31:43.458721+0000","last_undegraded":"2024-09-11T09:31:43.458721+0000","last_fullsized":"2024-09-11T09:31:42.702617+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.3","version":"17'1","reported_seq":347,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708630+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:43.708630+0000","last_peered":"2024-09-11T09:31:43.708630+0000","last_clean":"2024-09-11T09:31:43.708630+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:43.708630+0000","last_undegraded":"2024-09-11T09:31:43.708630+0000","last_fullsized":"2024-09-11T09:31:43.708630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":412,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704851+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:43.704851+0000","last_peered":"2024-09-11T09:31:43.704851+0000","last_clean":"2024-09-11T09:31:43.704851+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:43.704851+0000","last_undegraded":"2024-09-11T09:31:43.704851+0000","last_fullsized":"2024-09-11T09:31:43.704851+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13774,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700812+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:43.700812+0000","last_peered":"2024-09-11T09:31:43.700812+0000","last_clean":"2024-09-11T09:31:43.700812+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:43.700812+0000","last_undegraded":"2024-09-11T09:31:43.700812+0000","last_fullsized":"2024-09-11T09:31:43.700812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.9","version":"155'620","reported_seq":9280,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705176+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:43.705176+0000","last_peered":"2024-09-11T09:31:43.705176+0000","last_clean":"2024-09-11T09:31:43.705176+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:43.705176+0000","last_undegraded":"2024-09-11T09:31:43.705176+0000","last_fullsized":"2024-09-11T09:31:43.705176+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.a","version":"152'817","reported_seq":8381,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.712412+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:43.712412+0000","last_peered":"2024-09-11T09:31:43.712412+0000","last_clean":"2024-09-11T09:31:43.712412+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:43.712412+0000","last_undegraded":"2024-09-11T09:31:43.712412+0000","last_fullsized":"2024-09-11T09:31:43.712412+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.b","version":"149'361","reported_seq":8246,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701611+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:43.701611+0000","last_peered":"2024-09-11T09:31:43.701611+0000","last_clean":"2024-09-11T09:31:43.701611+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:43.701611+0000","last_undegraded":"2024-09-11T09:31:43.701611+0000","last_fullsized":"2024-09-11T09:31:43.701611+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.c","version":"162'381","reported_seq":6585,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708606+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:43.708606+0000","last_peered":"2024-09-11T09:31:43.708606+0000","last_clean":"2024-09-11T09:31:43.708606+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:43.708606+0000","last_undegraded":"2024-09-11T09:31:43.708606+0000","last_fullsized":"2024-09-11T09:31:43.708606+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.d","version":"149'593","reported_seq":6515,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705236+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:43.705236+0000","last_peered":"2024-09-11T09:31:43.705236+0000","last_clean":"2024-09-11T09:31:43.705236+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:43.705236+0000","last_undegraded":"2024-09-11T09:31:43.705236+0000","last_fullsized":"2024-09-11T09:31:43.705236+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.e","version":"155'530","reported_seq":6846,"reported_epoch":173,"state":"peering","last_fresh":"2024-09-11T09:31:43.156448+0000","last_change":"2024-09-11T09:31:44.539057+0000","last_active":"2024-09-11T09:31:44.539057+0000","last_peered":"2024-09-11T09:31:43.156448+0000","last_clean":"2024-09-11T09:31:30.892653+0000","last_became_active":"2024-09-11T09:31:42.718456+0000","last_became_peered":"2024-09-11T09:31:42.718456+0000","last_unstale":"2024-09-11T09:31:43.156448+0000","last_undegraded":"2024-09-11T09:31:43.156448+0000","last_fullsized":"2024-09-11T09:31:42.697418+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.f","version":"152'459","reported_seq":13006,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708641+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:43.708641+0000","last_peered":"2024-09-11T09:31:43.708641+0000","last_clean":"2024-09-11T09:31:43.708641+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:43.708641+0000","last_undegraded":"2024-09-11T09:31:43.708641+0000","last_fullsized":"2024-09-11T09:31:43.708641+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.10","version":"151'841","reported_seq":13128,"reported_epoch":173,"state":"active+undersized+remapped","last_fresh":"2024-09-11T09:31:44.287578+0000","last_change":"2024-09-11T09:31:44.287578+0000","last_active":"2024-09-11T09:31:44.287578+0000","last_peered":"2024-09-11T09:31:44.287578+0000","last_clean":"2024-09-11T09:31:39.721122+0000","last_became_active":"2024-09-11T09:31:42.728695+0000","last_became_peered":"2024-09-11T09:31:42.728695+0000","last_unstale":"2024-09-11T09:31:44.287578+0000","last_undegraded":"2024-09-11T09:31:44.287578+0000","last_fullsized":"2024-09-11T09:31:42.696746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[0],"avail_no_missing":["0","1","5"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.11","version":"151'419","reported_seq":4803,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705102+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:43.705102+0000","last_peered":"2024-09-11T09:31:43.705102+0000","last_clean":"2024-09-11T09:31:43.705102+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:43.705102+0000","last_undegraded":"2024-09-11T09:31:43.705102+0000","last_fullsized":"2024-09-11T09:31:43.705102+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7668,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704857+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:43.704857+0000","last_peered":"2024-09-11T09:31:43.704857+0000","last_clean":"2024-09-11T09:31:43.704857+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:43.704857+0000","last_undegraded":"2024-09-11T09:31:43.704857+0000","last_fullsized":"2024-09-11T09:31:43.704857+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607596,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73472,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298368,"statfs":{"total":96636764160,"available":96561528832,"internally_reserved":0,"allocated":2076672,"data_stored":881576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607595,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77628,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294212,"statfs":{"total":96636764160,"available":96557273088,"internally_reserved":0,"allocated":2064384,"data_stored":876429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59616,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312224,"statfs":{"total":96636764160,"available":96575717376,"internally_reserved":0,"allocated":2101248,"data_stored":1466778,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607596,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29060,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342780,"statfs":{"total":96636764160,"available":96607006720,"internally_reserved":0,"allocated":2072576,"data_stored":881557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77644,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294196,"statfs":{"total":96636764160,"available":96557256704,"internally_reserved":0,"allocated":2080768,"data_stored":881595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:46.940 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:47.228 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:47.239 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-11T09:31:46.755829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352812,"kb_used_data":12244,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878228,"statfs":{"total":579820584960,"available":579459305472,"internally_reserved":0,"allocated":12537856,"data_stored":6464916,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1485,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.060111"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7159,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701106+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:43.701106+0000","last_peered":"2024-09-11T09:31:43.701106+0000","last_clean":"2024-09-11T09:31:43.701106+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:43.701106+0000","last_undegraded":"2024-09-11T09:31:43.701106+0000","last_fullsized":"2024-09-11T09:31:43.701106+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1d","version":"155'718","reported_seq":7972,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708625+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:43.708625+0000","last_peered":"2024-09-11T09:31:43.708625+0000","last_clean":"2024-09-11T09:31:43.708625+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:43.708625+0000","last_undegraded":"2024-09-11T09:31:43.708625+0000","last_fullsized":"2024-09-11T09:31:43.708625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1c","version":"155'740","reported_seq":14240,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708709+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:43.708709+0000","last_peered":"2024-09-11T09:31:43.708709+0000","last_clean":"2024-09-11T09:31:43.708709+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:43.708709+0000","last_undegraded":"2024-09-11T09:31:43.708709+0000","last_fullsized":"2024-09-11T09:31:43.708709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.711685+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:43.711685+0000","last_peered":"2024-09-11T09:31:43.711685+0000","last_clean":"2024-09-11T09:31:43.711685+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:43.711685+0000","last_undegraded":"2024-09-11T09:31:43.711685+0000","last_fullsized":"2024-09-11T09:31:43.711685+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.19","version":"151'697","reported_seq":8418,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708084+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:43.708084+0000","last_peered":"2024-09-11T09:31:43.708084+0000","last_clean":"2024-09-11T09:31:43.708084+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:43.708084+0000","last_undegraded":"2024-09-11T09:31:43.708084+0000","last_fullsized":"2024-09-11T09:31:43.708084+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.18","version":"151'490","reported_seq":3311,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700942+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:43.700942+0000","last_peered":"2024-09-11T09:31:43.700942+0000","last_clean":"2024-09-11T09:31:43.700942+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:43.700942+0000","last_undegraded":"2024-09-11T09:31:43.700942+0000","last_fullsized":"2024-09-11T09:31:43.700942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10800,"reported_epoch":173,"state":"active+remapped","last_fresh":"2024-09-11T09:31:43.957784+0000","last_change":"2024-09-11T09:31:43.958006+0000","last_active":"2024-09-11T09:31:43.957784+0000","last_peered":"2024-09-11T09:31:43.957784+0000","last_clean":"2024-09-11T09:31:36.719409+0000","last_became_active":"2024-09-11T09:31:42.729993+0000","last_became_peered":"2024-09-11T09:31:42.729993+0000","last_unstale":"2024-09-11T09:31:43.957784+0000","last_undegraded":"2024-09-11T09:31:43.957784+0000","last_fullsized":"2024-09-11T09:31:43.957784+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[1,3],"avail_no_missing":["1","3","5"],"object_location_counts":[{"shards":"1,3","objects":12}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.15","version":"157'602","reported_seq":6288,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705415+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:43.705415+0000","last_peered":"2024-09-11T09:31:43.705415+0000","last_clean":"2024-09-11T09:31:43.705415+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:43.705415+0000","last_undegraded":"2024-09-11T09:31:43.705415+0000","last_fullsized":"2024-09-11T09:31:43.705415+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.14","version":"151'682","reported_seq":8990,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705108+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:43.705108+0000","last_peered":"2024-09-11T09:31:43.705108+0000","last_clean":"2024-09-11T09:31:43.705108+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:43.705108+0000","last_undegraded":"2024-09-11T09:31:43.705108+0000","last_fullsized":"2024-09-11T09:31:43.705108+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704976+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:43.704976+0000","last_peered":"2024-09-11T09:31:43.704976+0000","last_clean":"2024-09-11T09:31:43.704976+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:43.704976+0000","last_undegraded":"2024-09-11T09:31:43.704976+0000","last_fullsized":"2024-09-11T09:31:43.704976+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701734+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:43.701734+0000","last_peered":"2024-09-11T09:31:43.701734+0000","last_clean":"2024-09-11T09:31:43.701734+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:43.701734+0000","last_undegraded":"2024-09-11T09:31:43.701734+0000","last_fullsized":"2024-09-11T09:31:43.701734+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7482,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701333+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:43.701333+0000","last_peered":"2024-09-11T09:31:43.701333+0000","last_clean":"2024-09-11T09:31:43.701333+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:43.701333+0000","last_undegraded":"2024-09-11T09:31:43.701333+0000","last_fullsized":"2024-09-11T09:31:43.701333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705331+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:43.705331+0000","last_peered":"2024-09-11T09:31:43.705331+0000","last_clean":"2024-09-11T09:31:43.705331+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:43.705331+0000","last_undegraded":"2024-09-11T09:31:43.705331+0000","last_fullsized":"2024-09-11T09:31:43.705331+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7103,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701609+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:43.701609+0000","last_peered":"2024-09-11T09:31:43.701609+0000","last_clean":"2024-09-11T09:31:43.701609+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:43.701609+0000","last_undegraded":"2024-09-11T09:31:43.701609+0000","last_fullsized":"2024-09-11T09:31:43.701609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.4","version":"0'0","reported_seq":287,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701664+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:43.701664+0000","last_peered":"2024-09-11T09:31:43.701664+0000","last_clean":"2024-09-11T09:31:43.701664+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:43.701664+0000","last_undegraded":"2024-09-11T09:31:43.701664+0000","last_fullsized":"2024-09-11T09:31:43.701664+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5963,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708753+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:43.708753+0000","last_peered":"2024-09-11T09:31:43.708753+0000","last_clean":"2024-09-11T09:31:43.708753+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:43.708753+0000","last_undegraded":"2024-09-11T09:31:43.708753+0000","last_fullsized":"2024-09-11T09:31:43.708753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.3","version":"165'990","reported_seq":13947,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704963+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:43.704963+0000","last_peered":"2024-09-11T09:31:43.704963+0000","last_clean":"2024-09-11T09:31:43.704963+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:43.704963+0000","last_undegraded":"2024-09-11T09:31:43.704963+0000","last_fullsized":"2024-09-11T09:31:43.704963+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.2","version":"19'2","reported_seq":330,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708278+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:43.708278+0000","last_peered":"2024-09-11T09:31:43.708278+0000","last_clean":"2024-09-11T09:31:43.708278+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:43.708278+0000","last_undegraded":"2024-09-11T09:31:43.708278+0000","last_fullsized":"2024-09-11T09:31:43.708278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11644,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701669+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:43.701669+0000","last_peered":"2024-09-11T09:31:43.701669+0000","last_clean":"2024-09-11T09:31:43.701669+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:43.701669+0000","last_undegraded":"2024-09-11T09:31:43.701669+0000","last_fullsized":"2024-09-11T09:31:43.701669+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700944+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:43.700944+0000","last_peered":"2024-09-11T09:31:43.700944+0000","last_clean":"2024-09-11T09:31:43.700944+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:43.700944+0000","last_undegraded":"2024-09-11T09:31:43.700944+0000","last_fullsized":"2024-09-11T09:31:43.700944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":213,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704751+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:43.704751+0000","last_peered":"2024-09-11T09:31:43.704751+0000","last_clean":"2024-09-11T09:31:43.704751+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:43.704751+0000","last_undegraded":"2024-09-11T09:31:43.704751+0000","last_fullsized":"2024-09-11T09:31:43.704751+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":5998,"reported_epoch":172,"state":"peering","last_fresh":"2024-09-11T09:31:43.458721+0000","last_change":"2024-09-11T09:31:43.702628+0000","last_active":"2024-09-11T09:31:43.702628+0000","last_peered":"2024-09-11T09:31:43.458721+0000","last_clean":"2024-09-11T09:31:39.719720+0000","last_became_active":"2024-09-11T09:31:42.722345+0000","last_became_peered":"2024-09-11T09:31:42.722345+0000","last_unstale":"2024-09-11T09:31:43.458721+0000","last_undegraded":"2024-09-11T09:31:43.458721+0000","last_fullsized":"2024-09-11T09:31:42.702617+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.3","version":"17'1","reported_seq":347,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708630+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:43.708630+0000","last_peered":"2024-09-11T09:31:43.708630+0000","last_clean":"2024-09-11T09:31:43.708630+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:43.708630+0000","last_undegraded":"2024-09-11T09:31:43.708630+0000","last_fullsized":"2024-09-11T09:31:43.708630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":412,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704851+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:43.704851+0000","last_peered":"2024-09-11T09:31:43.704851+0000","last_clean":"2024-09-11T09:31:43.704851+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:43.704851+0000","last_undegraded":"2024-09-11T09:31:43.704851+0000","last_fullsized":"2024-09-11T09:31:43.704851+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13774,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700812+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:43.700812+0000","last_peered":"2024-09-11T09:31:43.700812+0000","last_clean":"2024-09-11T09:31:43.700812+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:43.700812+0000","last_undegraded":"2024-09-11T09:31:43.700812+0000","last_fullsized":"2024-09-11T09:31:43.700812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.9","version":"155'620","reported_seq":9280,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705176+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:43.705176+0000","last_peered":"2024-09-11T09:31:43.705176+0000","last_clean":"2024-09-11T09:31:43.705176+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:43.705176+0000","last_undegraded":"2024-09-11T09:31:43.705176+0000","last_fullsized":"2024-09-11T09:31:43.705176+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.a","version":"152'817","reported_seq":8381,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.712412+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:43.712412+0000","last_peered":"2024-09-11T09:31:43.712412+0000","last_clean":"2024-09-11T09:31:43.712412+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:43.712412+0000","last_undegraded":"2024-09-11T09:31:43.712412+0000","last_fullsized":"2024-09-11T09:31:43.712412+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.b","version":"149'361","reported_seq":8246,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701611+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:43.701611+0000","last_peered":"2024-09-11T09:31:43.701611+0000","last_clean":"2024-09-11T09:31:43.701611+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:43.701611+0000","last_undegraded":"2024-09-11T09:31:43.701611+0000","last_fullsized":"2024-09-11T09:31:43.701611+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.c","version":"162'381","reported_seq":6585,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708606+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:43.708606+0000","last_peered":"2024-09-11T09:31:43.708606+0000","last_clean":"2024-09-11T09:31:43.708606+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:43.708606+0000","last_undegraded":"2024-09-11T09:31:43.708606+0000","last_fullsized":"2024-09-11T09:31:43.708606+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.d","version":"149'593","reported_seq":6515,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705236+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:43.705236+0000","last_peered":"2024-09-11T09:31:43.705236+0000","last_clean":"2024-09-11T09:31:43.705236+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:43.705236+0000","last_undegraded":"2024-09-11T09:31:43.705236+0000","last_fullsized":"2024-09-11T09:31:43.705236+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.e","version":"155'530","reported_seq":6846,"reported_epoch":173,"state":"peering","last_fresh":"2024-09-11T09:31:43.156448+0000","last_change":"2024-09-11T09:31:44.539057+0000","last_active":"2024-09-11T09:31:44.539057+0000","last_peered":"2024-09-11T09:31:43.156448+0000","last_clean":"2024-09-11T09:31:30.892653+0000","last_became_active":"2024-09-11T09:31:42.718456+0000","last_became_peered":"2024-09-11T09:31:42.718456+0000","last_unstale":"2024-09-11T09:31:43.156448+0000","last_undegraded":"2024-09-11T09:31:43.156448+0000","last_fullsized":"2024-09-11T09:31:42.697418+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.f","version":"152'459","reported_seq":13006,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708641+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:43.708641+0000","last_peered":"2024-09-11T09:31:43.708641+0000","last_clean":"2024-09-11T09:31:43.708641+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:43.708641+0000","last_undegraded":"2024-09-11T09:31:43.708641+0000","last_fullsized":"2024-09-11T09:31:43.708641+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.10","version":"151'841","reported_seq":13128,"reported_epoch":173,"state":"active+undersized+remapped","last_fresh":"2024-09-11T09:31:44.287578+0000","last_change":"2024-09-11T09:31:44.287578+0000","last_active":"2024-09-11T09:31:44.287578+0000","last_peered":"2024-09-11T09:31:44.287578+0000","last_clean":"2024-09-11T09:31:39.721122+0000","last_became_active":"2024-09-11T09:31:42.728695+0000","last_became_peered":"2024-09-11T09:31:42.728695+0000","last_unstale":"2024-09-11T09:31:44.287578+0000","last_undegraded":"2024-09-11T09:31:44.287578+0000","last_fullsized":"2024-09-11T09:31:42.696746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[0],"avail_no_missing":["0","1","5"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.11","version":"151'419","reported_seq":4803,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705102+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:43.705102+0000","last_peered":"2024-09-11T09:31:43.705102+0000","last_clean":"2024-09-11T09:31:43.705102+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:43.705102+0000","last_undegraded":"2024-09-11T09:31:43.705102+0000","last_fullsized":"2024-09-11T09:31:43.705102+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7668,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704857+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:43.704857+0000","last_peered":"2024-09-11T09:31:43.704857+0000","last_clean":"2024-09-11T09:31:43.704857+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:43.704857+0000","last_undegraded":"2024-09-11T09:31:43.704857+0000","last_fullsized":"2024-09-11T09:31:43.704857+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607596,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73472,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298368,"statfs":{"total":96636764160,"available":96561528832,"internally_reserved":0,"allocated":2076672,"data_stored":881576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607595,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77628,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294212,"statfs":{"total":96636764160,"available":96557273088,"internally_reserved":0,"allocated":2064384,"data_stored":876429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59616,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312224,"statfs":{"total":96636764160,"available":96575717376,"internally_reserved":0,"allocated":2101248,"data_stored":1466778,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607596,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29060,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342780,"statfs":{"total":96636764160,"available":96607006720,"internally_reserved":0,"allocated":2072576,"data_stored":881557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77644,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294196,"statfs":{"total":96636764160,"available":96557256704,"internally_reserved":0,"allocated":2080768,"data_stored":881595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:47.242 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T09:31:47.593 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:47.594 INFO:teuthology.orchestra.run.smithi017.stdout:{"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":209,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":175,"num_osds":6,"num_up_osds":6,"osd_up_since":1726047091,"num_in_osds":6,"osd_in_since":1726047091,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":37},{"state_name":"peering","count":2},{"state_name":"active+remapped","count":1},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":405,"data_bytes":590406,"bytes_used":361279488,"bytes_avail":579459305472,"bytes_total":579820584960,"inactive_pgs_ratio":0.048780485987663269,"recovering_objects_per_sec":365,"recovering_bytes_per_sec":4,"recovering_keys_per_sec":0,"num_objects_recovered":1485,"num_bytes_recovered":19,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T09:28:17:886889+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T09:29:28.730550+0000","services":{}},"progress_events":{}} 2024-09-11T09:31:47.606 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-11T09:31:47.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:47.895 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:47.895 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:47.907 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-11T09:31:47.696618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352812,"kb_used_data":12244,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878228,"statfs":{"total":579820584960,"available":579459305472,"internally_reserved":0,"allocated":12537856,"data_stored":6464916,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1485,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.941453"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7159,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701106+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:43.701106+0000","last_peered":"2024-09-11T09:31:43.701106+0000","last_clean":"2024-09-11T09:31:43.701106+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:43.701106+0000","last_undegraded":"2024-09-11T09:31:43.701106+0000","last_fullsized":"2024-09-11T09:31:43.701106+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1d","version":"155'718","reported_seq":7972,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708625+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:43.708625+0000","last_peered":"2024-09-11T09:31:43.708625+0000","last_clean":"2024-09-11T09:31:43.708625+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:43.708625+0000","last_undegraded":"2024-09-11T09:31:43.708625+0000","last_fullsized":"2024-09-11T09:31:43.708625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1c","version":"155'740","reported_seq":14240,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708709+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:43.708709+0000","last_peered":"2024-09-11T09:31:43.708709+0000","last_clean":"2024-09-11T09:31:43.708709+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:43.708709+0000","last_undegraded":"2024-09-11T09:31:43.708709+0000","last_fullsized":"2024-09-11T09:31:43.708709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9167,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.711685+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:43.711685+0000","last_peered":"2024-09-11T09:31:43.711685+0000","last_clean":"2024-09-11T09:31:43.711685+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:43.711685+0000","last_undegraded":"2024-09-11T09:31:43.711685+0000","last_fullsized":"2024-09-11T09:31:43.711685+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.19","version":"151'697","reported_seq":8418,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708084+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:43.708084+0000","last_peered":"2024-09-11T09:31:43.708084+0000","last_clean":"2024-09-11T09:31:43.708084+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:43.708084+0000","last_undegraded":"2024-09-11T09:31:43.708084+0000","last_fullsized":"2024-09-11T09:31:43.708084+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.18","version":"151'490","reported_seq":3311,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700942+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:43.700942+0000","last_peered":"2024-09-11T09:31:43.700942+0000","last_clean":"2024-09-11T09:31:43.700942+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:43.700942+0000","last_undegraded":"2024-09-11T09:31:43.700942+0000","last_fullsized":"2024-09-11T09:31:43.700942+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10800,"reported_epoch":173,"state":"active+remapped","last_fresh":"2024-09-11T09:31:43.957784+0000","last_change":"2024-09-11T09:31:43.958006+0000","last_active":"2024-09-11T09:31:43.957784+0000","last_peered":"2024-09-11T09:31:43.957784+0000","last_clean":"2024-09-11T09:31:36.719409+0000","last_became_active":"2024-09-11T09:31:42.729993+0000","last_became_peered":"2024-09-11T09:31:42.729993+0000","last_unstale":"2024-09-11T09:31:43.957784+0000","last_undegraded":"2024-09-11T09:31:43.957784+0000","last_fullsized":"2024-09-11T09:31:43.957784+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[1,3],"avail_no_missing":["1","3","5"],"object_location_counts":[{"shards":"1,3","objects":12}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.15","version":"157'602","reported_seq":6288,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705415+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:43.705415+0000","last_peered":"2024-09-11T09:31:43.705415+0000","last_clean":"2024-09-11T09:31:43.705415+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:43.705415+0000","last_undegraded":"2024-09-11T09:31:43.705415+0000","last_fullsized":"2024-09-11T09:31:43.705415+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.14","version":"151'682","reported_seq":8990,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705108+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:43.705108+0000","last_peered":"2024-09-11T09:31:43.705108+0000","last_clean":"2024-09-11T09:31:43.705108+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:43.705108+0000","last_undegraded":"2024-09-11T09:31:43.705108+0000","last_fullsized":"2024-09-11T09:31:43.705108+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.7","version":"0'0","reported_seq":97,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704976+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:43.704976+0000","last_peered":"2024-09-11T09:31:43.704976+0000","last_clean":"2024-09-11T09:31:43.704976+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:43.704976+0000","last_undegraded":"2024-09-11T09:31:43.704976+0000","last_fullsized":"2024-09-11T09:31:43.704976+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701734+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:43.701734+0000","last_peered":"2024-09-11T09:31:43.701734+0000","last_clean":"2024-09-11T09:31:43.701734+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:43.701734+0000","last_undegraded":"2024-09-11T09:31:43.701734+0000","last_fullsized":"2024-09-11T09:31:43.701734+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7482,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701333+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:43.701333+0000","last_peered":"2024-09-11T09:31:43.701333+0000","last_clean":"2024-09-11T09:31:43.701333+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:43.701333+0000","last_undegraded":"2024-09-11T09:31:43.701333+0000","last_fullsized":"2024-09-11T09:31:43.701333+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.5","version":"0'0","reported_seq":209,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705331+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:43.705331+0000","last_peered":"2024-09-11T09:31:43.705331+0000","last_clean":"2024-09-11T09:31:43.705331+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:43.705331+0000","last_undegraded":"2024-09-11T09:31:43.705331+0000","last_fullsized":"2024-09-11T09:31:43.705331+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7103,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701609+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:43.701609+0000","last_peered":"2024-09-11T09:31:43.701609+0000","last_clean":"2024-09-11T09:31:43.701609+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:43.701609+0000","last_undegraded":"2024-09-11T09:31:43.701609+0000","last_fullsized":"2024-09-11T09:31:43.701609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.4","version":"0'0","reported_seq":287,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701664+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:43.701664+0000","last_peered":"2024-09-11T09:31:43.701664+0000","last_clean":"2024-09-11T09:31:43.701664+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:43.701664+0000","last_undegraded":"2024-09-11T09:31:43.701664+0000","last_fullsized":"2024-09-11T09:31:43.701664+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5963,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708753+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:43.708753+0000","last_peered":"2024-09-11T09:31:43.708753+0000","last_clean":"2024-09-11T09:31:43.708753+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:43.708753+0000","last_undegraded":"2024-09-11T09:31:43.708753+0000","last_fullsized":"2024-09-11T09:31:43.708753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.3","version":"165'990","reported_seq":13947,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704963+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:43.704963+0000","last_peered":"2024-09-11T09:31:43.704963+0000","last_clean":"2024-09-11T09:31:43.704963+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:43.704963+0000","last_undegraded":"2024-09-11T09:31:43.704963+0000","last_fullsized":"2024-09-11T09:31:43.704963+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.2","version":"19'2","reported_seq":330,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708278+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:43.708278+0000","last_peered":"2024-09-11T09:31:43.708278+0000","last_clean":"2024-09-11T09:31:43.708278+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:43.708278+0000","last_undegraded":"2024-09-11T09:31:43.708278+0000","last_fullsized":"2024-09-11T09:31:43.708278+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11644,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701669+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:43.701669+0000","last_peered":"2024-09-11T09:31:43.701669+0000","last_clean":"2024-09-11T09:31:43.701669+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:43.701669+0000","last_undegraded":"2024-09-11T09:31:43.701669+0000","last_fullsized":"2024-09-11T09:31:43.701669+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.1","version":"0'0","reported_seq":210,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700944+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:43.700944+0000","last_peered":"2024-09-11T09:31:43.700944+0000","last_clean":"2024-09-11T09:31:43.700944+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:43.700944+0000","last_undegraded":"2024-09-11T09:31:43.700944+0000","last_fullsized":"2024-09-11T09:31:43.700944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":213,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704751+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:43.704751+0000","last_peered":"2024-09-11T09:31:43.704751+0000","last_clean":"2024-09-11T09:31:43.704751+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:43.704751+0000","last_undegraded":"2024-09-11T09:31:43.704751+0000","last_fullsized":"2024-09-11T09:31:43.704751+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":5998,"reported_epoch":172,"state":"peering","last_fresh":"2024-09-11T09:31:43.458721+0000","last_change":"2024-09-11T09:31:43.702628+0000","last_active":"2024-09-11T09:31:43.702628+0000","last_peered":"2024-09-11T09:31:43.458721+0000","last_clean":"2024-09-11T09:31:39.719720+0000","last_became_active":"2024-09-11T09:31:42.722345+0000","last_became_peered":"2024-09-11T09:31:42.722345+0000","last_unstale":"2024-09-11T09:31:43.458721+0000","last_undegraded":"2024-09-11T09:31:43.458721+0000","last_fullsized":"2024-09-11T09:31:42.702617+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.3","version":"17'1","reported_seq":347,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708630+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:43.708630+0000","last_peered":"2024-09-11T09:31:43.708630+0000","last_clean":"2024-09-11T09:31:43.708630+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:43.708630+0000","last_undegraded":"2024-09-11T09:31:43.708630+0000","last_fullsized":"2024-09-11T09:31:43.708630+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":412,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704851+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:43.704851+0000","last_peered":"2024-09-11T09:31:43.704851+0000","last_clean":"2024-09-11T09:31:43.704851+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:43.704851+0000","last_undegraded":"2024-09-11T09:31:43.704851+0000","last_fullsized":"2024-09-11T09:31:43.704851+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13774,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.700812+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:43.700812+0000","last_peered":"2024-09-11T09:31:43.700812+0000","last_clean":"2024-09-11T09:31:43.700812+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:43.700812+0000","last_undegraded":"2024-09-11T09:31:43.700812+0000","last_fullsized":"2024-09-11T09:31:43.700812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.9","version":"155'620","reported_seq":9280,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705176+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:43.705176+0000","last_peered":"2024-09-11T09:31:43.705176+0000","last_clean":"2024-09-11T09:31:43.705176+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:43.705176+0000","last_undegraded":"2024-09-11T09:31:43.705176+0000","last_fullsized":"2024-09-11T09:31:43.705176+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.a","version":"152'817","reported_seq":8381,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.712412+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:43.712412+0000","last_peered":"2024-09-11T09:31:43.712412+0000","last_clean":"2024-09-11T09:31:43.712412+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:43.712412+0000","last_undegraded":"2024-09-11T09:31:43.712412+0000","last_fullsized":"2024-09-11T09:31:43.712412+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.b","version":"149'361","reported_seq":8246,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.701611+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:43.701611+0000","last_peered":"2024-09-11T09:31:43.701611+0000","last_clean":"2024-09-11T09:31:43.701611+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:43.701611+0000","last_undegraded":"2024-09-11T09:31:43.701611+0000","last_fullsized":"2024-09-11T09:31:43.701611+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.c","version":"162'381","reported_seq":6585,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708606+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:43.708606+0000","last_peered":"2024-09-11T09:31:43.708606+0000","last_clean":"2024-09-11T09:31:43.708606+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:43.708606+0000","last_undegraded":"2024-09-11T09:31:43.708606+0000","last_fullsized":"2024-09-11T09:31:43.708606+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.d","version":"149'593","reported_seq":6515,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705236+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:43.705236+0000","last_peered":"2024-09-11T09:31:43.705236+0000","last_clean":"2024-09-11T09:31:43.705236+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:43.705236+0000","last_undegraded":"2024-09-11T09:31:43.705236+0000","last_fullsized":"2024-09-11T09:31:43.705236+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.e","version":"155'530","reported_seq":6846,"reported_epoch":173,"state":"peering","last_fresh":"2024-09-11T09:31:43.156448+0000","last_change":"2024-09-11T09:31:44.539057+0000","last_active":"2024-09-11T09:31:44.539057+0000","last_peered":"2024-09-11T09:31:43.156448+0000","last_clean":"2024-09-11T09:31:30.892653+0000","last_became_active":"2024-09-11T09:31:42.718456+0000","last_became_peered":"2024-09-11T09:31:42.718456+0000","last_unstale":"2024-09-11T09:31:43.156448+0000","last_undegraded":"2024-09-11T09:31:43.156448+0000","last_fullsized":"2024-09-11T09:31:42.697418+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.f","version":"152'459","reported_seq":13006,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.708641+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:43.708641+0000","last_peered":"2024-09-11T09:31:43.708641+0000","last_clean":"2024-09-11T09:31:43.708641+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:43.708641+0000","last_undegraded":"2024-09-11T09:31:43.708641+0000","last_fullsized":"2024-09-11T09:31:43.708641+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.10","version":"151'841","reported_seq":13128,"reported_epoch":173,"state":"active+undersized+remapped","last_fresh":"2024-09-11T09:31:44.287578+0000","last_change":"2024-09-11T09:31:44.287578+0000","last_active":"2024-09-11T09:31:44.287578+0000","last_peered":"2024-09-11T09:31:44.287578+0000","last_clean":"2024-09-11T09:31:39.721122+0000","last_became_active":"2024-09-11T09:31:42.728695+0000","last_became_peered":"2024-09-11T09:31:42.728695+0000","last_unstale":"2024-09-11T09:31:44.287578+0000","last_undegraded":"2024-09-11T09:31:44.287578+0000","last_fullsized":"2024-09-11T09:31:42.696746+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[0],"avail_no_missing":["0","1","5"],"object_location_counts":[{"shards":"0","objects":8}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.11","version":"151'419","reported_seq":4803,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.705102+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:43.705102+0000","last_peered":"2024-09-11T09:31:43.705102+0000","last_clean":"2024-09-11T09:31:43.705102+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:43.705102+0000","last_undegraded":"2024-09-11T09:31:43.705102+0000","last_fullsized":"2024-09-11T09:31:43.705102+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7668,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-11T09:31:43.704857+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:43.704857+0000","last_peered":"2024-09-11T09:31:43.704857+0000","last_clean":"2024-09-11T09:31:43.704857+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:43.704857+0000","last_undegraded":"2024-09-11T09:31:43.704857+0000","last_fullsized":"2024-09-11T09:31:43.704857+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"2b"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":63,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607596,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73472,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298368,"statfs":{"total":96636764160,"available":96561528832,"internally_reserved":0,"allocated":2076672,"data_stored":881576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607595,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77628,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294212,"statfs":{"total":96636764160,"available":96557273088,"internally_reserved":0,"allocated":2064384,"data_stored":876429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59616,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312224,"statfs":{"total":96636764160,"available":96575717376,"internally_reserved":0,"allocated":2101248,"data_stored":1466778,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607596,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29060,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342780,"statfs":{"total":96636764160,"available":96607006720,"internally_reserved":0,"allocated":2072576,"data_stored":881557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77644,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294196,"statfs":{"total":96636764160,"available":96557256704,"internally_reserved":0,"allocated":2080768,"data_stored":881595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:50.910 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:51.214 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:51.215 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:51.226 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T09:31:50.756845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352968,"kb_used_data":12412,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878072,"statfs":{"total":579820584960,"available":579459145728,"internally_reserved":0,"allocated":12709888,"data_stored":6535516,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.052730"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7165,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697661+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:47.697661+0000","last_peered":"2024-09-11T09:31:47.697661+0000","last_clean":"2024-09-11T09:31:47.697661+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:47.697661+0000","last_undegraded":"2024-09-11T09:31:47.697661+0000","last_fullsized":"2024-09-11T09:31:47.697661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"155'718","reported_seq":7978,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006622+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:48.006622+0000","last_peered":"2024-09-11T09:31:48.006622+0000","last_clean":"2024-09-11T09:31:48.006622+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:48.006622+0000","last_undegraded":"2024-09-11T09:31:48.006622+0000","last_fullsized":"2024-09-11T09:31:48.006622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"155'740","reported_seq":14246,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006809+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:48.006809+0000","last_peered":"2024-09-11T09:31:48.006809+0000","last_clean":"2024-09-11T09:31:48.006809+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:48.006809+0000","last_undegraded":"2024-09-11T09:31:48.006809+0000","last_fullsized":"2024-09-11T09:31:48.006809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006282+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:48.006282+0000","last_peered":"2024-09-11T09:31:48.006282+0000","last_clean":"2024-09-11T09:31:48.006282+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:48.006282+0000","last_undegraded":"2024-09-11T09:31:48.006282+0000","last_fullsized":"2024-09-11T09:31:48.006282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"151'697","reported_seq":8424,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005465+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:48.005465+0000","last_peered":"2024-09-11T09:31:48.005465+0000","last_clean":"2024-09-11T09:31:48.005465+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:48.005465+0000","last_undegraded":"2024-09-11T09:31:48.005465+0000","last_fullsized":"2024-09-11T09:31:48.005465+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"151'490","reported_seq":3317,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697747+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:47.697747+0000","last_peered":"2024-09-11T09:31:47.697747+0000","last_clean":"2024-09-11T09:31:47.697747+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:47.697747+0000","last_undegraded":"2024-09-11T09:31:47.697747+0000","last_fullsized":"2024-09-11T09:31:47.697747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10534,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007045+0000","last_change":"2024-09-11T09:31:45.711457+0000","last_active":"2024-09-11T09:31:48.007045+0000","last_peered":"2024-09-11T09:31:48.007045+0000","last_clean":"2024-09-11T09:31:48.007045+0000","last_became_active":"2024-09-11T09:31:45.709608+0000","last_became_peered":"2024-09-11T09:31:45.709608+0000","last_unstale":"2024-09-11T09:31:48.007045+0000","last_undegraded":"2024-09-11T09:31:48.007045+0000","last_fullsized":"2024-09-11T09:31:48.007045+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:31:35.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'602","reported_seq":6294,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698462+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:47.698462+0000","last_peered":"2024-09-11T09:31:47.698462+0000","last_clean":"2024-09-11T09:31:47.698462+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:47.698462+0000","last_undegraded":"2024-09-11T09:31:47.698462+0000","last_fullsized":"2024-09-11T09:31:47.698462+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"151'682","reported_seq":8996,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698307+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:47.698307+0000","last_peered":"2024-09-11T09:31:47.698307+0000","last_clean":"2024-09-11T09:31:47.698307+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:47.698307+0000","last_undegraded":"2024-09-11T09:31:47.698307+0000","last_fullsized":"2024-09-11T09:31:47.698307+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698629+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:47.698629+0000","last_peered":"2024-09-11T09:31:47.698629+0000","last_clean":"2024-09-11T09:31:47.698629+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:47.698629+0000","last_undegraded":"2024-09-11T09:31:47.698629+0000","last_fullsized":"2024-09-11T09:31:47.698629+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005498+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:48.005498+0000","last_peered":"2024-09-11T09:31:48.005498+0000","last_clean":"2024-09-11T09:31:48.005498+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:48.005498+0000","last_undegraded":"2024-09-11T09:31:48.005498+0000","last_fullsized":"2024-09-11T09:31:48.005498+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7488,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698027+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:47.698027+0000","last_peered":"2024-09-11T09:31:47.698027+0000","last_clean":"2024-09-11T09:31:47.698027+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:47.698027+0000","last_undegraded":"2024-09-11T09:31:47.698027+0000","last_fullsized":"2024-09-11T09:31:47.698027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698338+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:47.698338+0000","last_peered":"2024-09-11T09:31:47.698338+0000","last_clean":"2024-09-11T09:31:47.698338+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:47.698338+0000","last_undegraded":"2024-09-11T09:31:47.698338+0000","last_fullsized":"2024-09-11T09:31:47.698338+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7109,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005760+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:48.005760+0000","last_peered":"2024-09-11T09:31:48.005760+0000","last_clean":"2024-09-11T09:31:48.005760+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:48.005760+0000","last_undegraded":"2024-09-11T09:31:48.005760+0000","last_fullsized":"2024-09-11T09:31:48.005760+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":293,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005569+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:48.005569+0000","last_peered":"2024-09-11T09:31:48.005569+0000","last_clean":"2024-09-11T09:31:48.005569+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:48.005569+0000","last_undegraded":"2024-09-11T09:31:48.005569+0000","last_fullsized":"2024-09-11T09:31:48.005569+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5969,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007050+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:48.007050+0000","last_peered":"2024-09-11T09:31:48.007050+0000","last_clean":"2024-09-11T09:31:48.007050+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:48.007050+0000","last_undegraded":"2024-09-11T09:31:48.007050+0000","last_fullsized":"2024-09-11T09:31:48.007050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'990","reported_seq":13953,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006403+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:48.006403+0000","last_peered":"2024-09-11T09:31:48.006403+0000","last_clean":"2024-09-11T09:31:48.006403+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:48.006403+0000","last_undegraded":"2024-09-11T09:31:48.006403+0000","last_fullsized":"2024-09-11T09:31:48.006403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":336,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005987+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:48.005987+0000","last_peered":"2024-09-11T09:31:48.005987+0000","last_clean":"2024-09-11T09:31:48.005987+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:48.005987+0000","last_undegraded":"2024-09-11T09:31:48.005987+0000","last_fullsized":"2024-09-11T09:31:48.005987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005768+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:48.005768+0000","last_peered":"2024-09-11T09:31:48.005768+0000","last_clean":"2024-09-11T09:31:48.005768+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:48.005768+0000","last_undegraded":"2024-09-11T09:31:48.005768+0000","last_fullsized":"2024-09-11T09:31:48.005768+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698244+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:47.698244+0000","last_peered":"2024-09-11T09:31:47.698244+0000","last_clean":"2024-09-11T09:31:47.698244+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:47.698244+0000","last_undegraded":"2024-09-11T09:31:47.698244+0000","last_fullsized":"2024-09-11T09:31:47.698244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":219,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698143+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:47.698143+0000","last_peered":"2024-09-11T09:31:47.698143+0000","last_clean":"2024-09-11T09:31:47.698143+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:47.698143+0000","last_undegraded":"2024-09-11T09:31:47.698143+0000","last_fullsized":"2024-09-11T09:31:47.698143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":6010,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005747+0000","last_change":"2024-09-11T09:31:44.718923+0000","last_active":"2024-09-11T09:31:48.005747+0000","last_peered":"2024-09-11T09:31:48.005747+0000","last_clean":"2024-09-11T09:31:48.005747+0000","last_became_active":"2024-09-11T09:31:44.717222+0000","last_became_peered":"2024-09-11T09:31:44.717222+0000","last_unstale":"2024-09-11T09:31:48.005747+0000","last_undegraded":"2024-09-11T09:31:48.005747+0000","last_fullsized":"2024-09-11T09:31:48.005747+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:42.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":353,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006600+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:48.006600+0000","last_peered":"2024-09-11T09:31:48.006600+0000","last_clean":"2024-09-11T09:31:48.006600+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:48.006600+0000","last_undegraded":"2024-09-11T09:31:48.006600+0000","last_fullsized":"2024-09-11T09:31:48.006600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698175+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:47.698175+0000","last_peered":"2024-09-11T09:31:47.698175+0000","last_clean":"2024-09-11T09:31:47.698175+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:47.698175+0000","last_undegraded":"2024-09-11T09:31:47.698175+0000","last_fullsized":"2024-09-11T09:31:47.698175+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697540+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:47.697540+0000","last_peered":"2024-09-11T09:31:47.697540+0000","last_clean":"2024-09-11T09:31:47.697540+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:47.697540+0000","last_undegraded":"2024-09-11T09:31:47.697540+0000","last_fullsized":"2024-09-11T09:31:47.697540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"155'620","reported_seq":9286,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006280+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:48.006280+0000","last_peered":"2024-09-11T09:31:48.006280+0000","last_clean":"2024-09-11T09:31:48.006280+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:48.006280+0000","last_undegraded":"2024-09-11T09:31:48.006280+0000","last_fullsized":"2024-09-11T09:31:48.006280+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"152'817","reported_seq":8387,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698918+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:47.698918+0000","last_peered":"2024-09-11T09:31:47.698918+0000","last_clean":"2024-09-11T09:31:47.698918+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:47.698918+0000","last_undegraded":"2024-09-11T09:31:47.698918+0000","last_fullsized":"2024-09-11T09:31:47.698918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"149'361","reported_seq":8252,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005580+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:48.005580+0000","last_peered":"2024-09-11T09:31:48.005580+0000","last_clean":"2024-09-11T09:31:48.005580+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:48.005580+0000","last_undegraded":"2024-09-11T09:31:48.005580+0000","last_fullsized":"2024-09-11T09:31:48.005580+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'381","reported_seq":6591,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006368+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:48.006368+0000","last_peered":"2024-09-11T09:31:48.006368+0000","last_clean":"2024-09-11T09:31:48.006368+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:48.006368+0000","last_undegraded":"2024-09-11T09:31:48.006368+0000","last_fullsized":"2024-09-11T09:31:48.006368+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"149'593","reported_seq":6521,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698333+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:47.698333+0000","last_peered":"2024-09-11T09:31:47.698333+0000","last_clean":"2024-09-11T09:31:47.698333+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:47.698333+0000","last_undegraded":"2024-09-11T09:31:47.698333+0000","last_fullsized":"2024-09-11T09:31:47.698333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"155'530","reported_seq":6858,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006381+0000","last_change":"2024-09-11T09:31:44.711140+0000","last_active":"2024-09-11T09:31:48.006381+0000","last_peered":"2024-09-11T09:31:48.006381+0000","last_clean":"2024-09-11T09:31:48.006381+0000","last_became_active":"2024-09-11T09:31:44.709474+0000","last_became_peered":"2024-09-11T09:31:44.709474+0000","last_unstale":"2024-09-11T09:31:48.006381+0000","last_undegraded":"2024-09-11T09:31:48.006381+0000","last_fullsized":"2024-09-11T09:31:48.006381+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:08.396097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"152'459","reported_seq":13012,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006957+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:48.006957+0000","last_peered":"2024-09-11T09:31:48.006957+0000","last_clean":"2024-09-11T09:31:48.006957+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:48.006957+0000","last_undegraded":"2024-09-11T09:31:48.006957+0000","last_fullsized":"2024-09-11T09:31:48.006957+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"151'841","reported_seq":12855,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006161+0000","last_change":"2024-09-11T09:31:45.710037+0000","last_active":"2024-09-11T09:31:48.006161+0000","last_peered":"2024-09-11T09:31:48.006161+0000","last_clean":"2024-09-11T09:31:48.006161+0000","last_became_active":"2024-09-11T09:31:45.707966+0000","last_became_peered":"2024-09-11T09:31:45.707966+0000","last_unstale":"2024-09-11T09:31:48.006161+0000","last_undegraded":"2024-09-11T09:31:48.006161+0000","last_fullsized":"2024-09-11T09:31:48.006161+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:08.229335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"151'419","reported_seq":4809,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698278+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:47.698278+0000","last_peered":"2024-09-11T09:31:47.698278+0000","last_clean":"2024-09-11T09:31:47.698278+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:47.698278+0000","last_undegraded":"2024-09-11T09:31:47.698278+0000","last_fullsized":"2024-09-11T09:31:47.698278+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7674,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698421+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:47.698421+0000","last_peered":"2024-09-11T09:31:47.698421+0000","last_clean":"2024-09-11T09:31:47.698421+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:47.698421+0000","last_undegraded":"2024-09-11T09:31:47.698421+0000","last_fullsized":"2024-09-11T09:31:47.698421+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607597,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73508,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298332,"statfs":{"total":96636764160,"available":96561491968,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607596,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77664,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294176,"statfs":{"total":96636764160,"available":96557236224,"internally_reserved":0,"allocated":2101248,"data_stored":891630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2138112,"data_stored":1481979,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607597,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29080,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342760,"statfs":{"total":96636764160,"available":96606986240,"internally_reserved":0,"allocated":2097152,"data_stored":891611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607597,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":2117632,"data_stored":896667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:51.228 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-11T09:31:51.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-11T09:31:51.535 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T09:31:51.546 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T09:31:51.546 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:51.835 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:51.836 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:51.847 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T09:31:50.756845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352968,"kb_used_data":12412,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878072,"statfs":{"total":579820584960,"available":579459145728,"internally_reserved":0,"allocated":12709888,"data_stored":6535516,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.052730"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7165,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697661+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:47.697661+0000","last_peered":"2024-09-11T09:31:47.697661+0000","last_clean":"2024-09-11T09:31:47.697661+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:47.697661+0000","last_undegraded":"2024-09-11T09:31:47.697661+0000","last_fullsized":"2024-09-11T09:31:47.697661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"155'718","reported_seq":7978,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006622+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:48.006622+0000","last_peered":"2024-09-11T09:31:48.006622+0000","last_clean":"2024-09-11T09:31:48.006622+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:48.006622+0000","last_undegraded":"2024-09-11T09:31:48.006622+0000","last_fullsized":"2024-09-11T09:31:48.006622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"155'740","reported_seq":14246,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006809+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:48.006809+0000","last_peered":"2024-09-11T09:31:48.006809+0000","last_clean":"2024-09-11T09:31:48.006809+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:48.006809+0000","last_undegraded":"2024-09-11T09:31:48.006809+0000","last_fullsized":"2024-09-11T09:31:48.006809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006282+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:48.006282+0000","last_peered":"2024-09-11T09:31:48.006282+0000","last_clean":"2024-09-11T09:31:48.006282+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:48.006282+0000","last_undegraded":"2024-09-11T09:31:48.006282+0000","last_fullsized":"2024-09-11T09:31:48.006282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"151'697","reported_seq":8424,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005465+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:48.005465+0000","last_peered":"2024-09-11T09:31:48.005465+0000","last_clean":"2024-09-11T09:31:48.005465+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:48.005465+0000","last_undegraded":"2024-09-11T09:31:48.005465+0000","last_fullsized":"2024-09-11T09:31:48.005465+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"151'490","reported_seq":3317,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697747+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:47.697747+0000","last_peered":"2024-09-11T09:31:47.697747+0000","last_clean":"2024-09-11T09:31:47.697747+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:47.697747+0000","last_undegraded":"2024-09-11T09:31:47.697747+0000","last_fullsized":"2024-09-11T09:31:47.697747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10534,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007045+0000","last_change":"2024-09-11T09:31:45.711457+0000","last_active":"2024-09-11T09:31:48.007045+0000","last_peered":"2024-09-11T09:31:48.007045+0000","last_clean":"2024-09-11T09:31:48.007045+0000","last_became_active":"2024-09-11T09:31:45.709608+0000","last_became_peered":"2024-09-11T09:31:45.709608+0000","last_unstale":"2024-09-11T09:31:48.007045+0000","last_undegraded":"2024-09-11T09:31:48.007045+0000","last_fullsized":"2024-09-11T09:31:48.007045+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:31:35.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'602","reported_seq":6294,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698462+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:47.698462+0000","last_peered":"2024-09-11T09:31:47.698462+0000","last_clean":"2024-09-11T09:31:47.698462+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:47.698462+0000","last_undegraded":"2024-09-11T09:31:47.698462+0000","last_fullsized":"2024-09-11T09:31:47.698462+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"151'682","reported_seq":8996,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698307+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:47.698307+0000","last_peered":"2024-09-11T09:31:47.698307+0000","last_clean":"2024-09-11T09:31:47.698307+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:47.698307+0000","last_undegraded":"2024-09-11T09:31:47.698307+0000","last_fullsized":"2024-09-11T09:31:47.698307+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698629+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:47.698629+0000","last_peered":"2024-09-11T09:31:47.698629+0000","last_clean":"2024-09-11T09:31:47.698629+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:47.698629+0000","last_undegraded":"2024-09-11T09:31:47.698629+0000","last_fullsized":"2024-09-11T09:31:47.698629+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005498+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:48.005498+0000","last_peered":"2024-09-11T09:31:48.005498+0000","last_clean":"2024-09-11T09:31:48.005498+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:48.005498+0000","last_undegraded":"2024-09-11T09:31:48.005498+0000","last_fullsized":"2024-09-11T09:31:48.005498+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7488,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698027+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:47.698027+0000","last_peered":"2024-09-11T09:31:47.698027+0000","last_clean":"2024-09-11T09:31:47.698027+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:47.698027+0000","last_undegraded":"2024-09-11T09:31:47.698027+0000","last_fullsized":"2024-09-11T09:31:47.698027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698338+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:47.698338+0000","last_peered":"2024-09-11T09:31:47.698338+0000","last_clean":"2024-09-11T09:31:47.698338+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:47.698338+0000","last_undegraded":"2024-09-11T09:31:47.698338+0000","last_fullsized":"2024-09-11T09:31:47.698338+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7109,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005760+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:48.005760+0000","last_peered":"2024-09-11T09:31:48.005760+0000","last_clean":"2024-09-11T09:31:48.005760+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:48.005760+0000","last_undegraded":"2024-09-11T09:31:48.005760+0000","last_fullsized":"2024-09-11T09:31:48.005760+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":293,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005569+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:48.005569+0000","last_peered":"2024-09-11T09:31:48.005569+0000","last_clean":"2024-09-11T09:31:48.005569+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:48.005569+0000","last_undegraded":"2024-09-11T09:31:48.005569+0000","last_fullsized":"2024-09-11T09:31:48.005569+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5969,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007050+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:48.007050+0000","last_peered":"2024-09-11T09:31:48.007050+0000","last_clean":"2024-09-11T09:31:48.007050+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:48.007050+0000","last_undegraded":"2024-09-11T09:31:48.007050+0000","last_fullsized":"2024-09-11T09:31:48.007050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'990","reported_seq":13953,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006403+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:48.006403+0000","last_peered":"2024-09-11T09:31:48.006403+0000","last_clean":"2024-09-11T09:31:48.006403+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:48.006403+0000","last_undegraded":"2024-09-11T09:31:48.006403+0000","last_fullsized":"2024-09-11T09:31:48.006403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":336,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005987+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:48.005987+0000","last_peered":"2024-09-11T09:31:48.005987+0000","last_clean":"2024-09-11T09:31:48.005987+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:48.005987+0000","last_undegraded":"2024-09-11T09:31:48.005987+0000","last_fullsized":"2024-09-11T09:31:48.005987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005768+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:48.005768+0000","last_peered":"2024-09-11T09:31:48.005768+0000","last_clean":"2024-09-11T09:31:48.005768+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:48.005768+0000","last_undegraded":"2024-09-11T09:31:48.005768+0000","last_fullsized":"2024-09-11T09:31:48.005768+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698244+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:47.698244+0000","last_peered":"2024-09-11T09:31:47.698244+0000","last_clean":"2024-09-11T09:31:47.698244+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:47.698244+0000","last_undegraded":"2024-09-11T09:31:47.698244+0000","last_fullsized":"2024-09-11T09:31:47.698244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":219,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698143+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:47.698143+0000","last_peered":"2024-09-11T09:31:47.698143+0000","last_clean":"2024-09-11T09:31:47.698143+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:47.698143+0000","last_undegraded":"2024-09-11T09:31:47.698143+0000","last_fullsized":"2024-09-11T09:31:47.698143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":6010,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005747+0000","last_change":"2024-09-11T09:31:44.718923+0000","last_active":"2024-09-11T09:31:48.005747+0000","last_peered":"2024-09-11T09:31:48.005747+0000","last_clean":"2024-09-11T09:31:48.005747+0000","last_became_active":"2024-09-11T09:31:44.717222+0000","last_became_peered":"2024-09-11T09:31:44.717222+0000","last_unstale":"2024-09-11T09:31:48.005747+0000","last_undegraded":"2024-09-11T09:31:48.005747+0000","last_fullsized":"2024-09-11T09:31:48.005747+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:42.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":353,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006600+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:48.006600+0000","last_peered":"2024-09-11T09:31:48.006600+0000","last_clean":"2024-09-11T09:31:48.006600+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:48.006600+0000","last_undegraded":"2024-09-11T09:31:48.006600+0000","last_fullsized":"2024-09-11T09:31:48.006600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698175+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:47.698175+0000","last_peered":"2024-09-11T09:31:47.698175+0000","last_clean":"2024-09-11T09:31:47.698175+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:47.698175+0000","last_undegraded":"2024-09-11T09:31:47.698175+0000","last_fullsized":"2024-09-11T09:31:47.698175+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697540+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:47.697540+0000","last_peered":"2024-09-11T09:31:47.697540+0000","last_clean":"2024-09-11T09:31:47.697540+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:47.697540+0000","last_undegraded":"2024-09-11T09:31:47.697540+0000","last_fullsized":"2024-09-11T09:31:47.697540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"155'620","reported_seq":9286,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006280+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:48.006280+0000","last_peered":"2024-09-11T09:31:48.006280+0000","last_clean":"2024-09-11T09:31:48.006280+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:48.006280+0000","last_undegraded":"2024-09-11T09:31:48.006280+0000","last_fullsized":"2024-09-11T09:31:48.006280+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"152'817","reported_seq":8387,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698918+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:47.698918+0000","last_peered":"2024-09-11T09:31:47.698918+0000","last_clean":"2024-09-11T09:31:47.698918+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:47.698918+0000","last_undegraded":"2024-09-11T09:31:47.698918+0000","last_fullsized":"2024-09-11T09:31:47.698918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"149'361","reported_seq":8252,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005580+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:48.005580+0000","last_peered":"2024-09-11T09:31:48.005580+0000","last_clean":"2024-09-11T09:31:48.005580+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:48.005580+0000","last_undegraded":"2024-09-11T09:31:48.005580+0000","last_fullsized":"2024-09-11T09:31:48.005580+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'381","reported_seq":6591,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006368+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:48.006368+0000","last_peered":"2024-09-11T09:31:48.006368+0000","last_clean":"2024-09-11T09:31:48.006368+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:48.006368+0000","last_undegraded":"2024-09-11T09:31:48.006368+0000","last_fullsized":"2024-09-11T09:31:48.006368+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"149'593","reported_seq":6521,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698333+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:47.698333+0000","last_peered":"2024-09-11T09:31:47.698333+0000","last_clean":"2024-09-11T09:31:47.698333+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:47.698333+0000","last_undegraded":"2024-09-11T09:31:47.698333+0000","last_fullsized":"2024-09-11T09:31:47.698333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"155'530","reported_seq":6858,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006381+0000","last_change":"2024-09-11T09:31:44.711140+0000","last_active":"2024-09-11T09:31:48.006381+0000","last_peered":"2024-09-11T09:31:48.006381+0000","last_clean":"2024-09-11T09:31:48.006381+0000","last_became_active":"2024-09-11T09:31:44.709474+0000","last_became_peered":"2024-09-11T09:31:44.709474+0000","last_unstale":"2024-09-11T09:31:48.006381+0000","last_undegraded":"2024-09-11T09:31:48.006381+0000","last_fullsized":"2024-09-11T09:31:48.006381+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:08.396097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"152'459","reported_seq":13012,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006957+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:48.006957+0000","last_peered":"2024-09-11T09:31:48.006957+0000","last_clean":"2024-09-11T09:31:48.006957+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:48.006957+0000","last_undegraded":"2024-09-11T09:31:48.006957+0000","last_fullsized":"2024-09-11T09:31:48.006957+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"151'841","reported_seq":12855,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006161+0000","last_change":"2024-09-11T09:31:45.710037+0000","last_active":"2024-09-11T09:31:48.006161+0000","last_peered":"2024-09-11T09:31:48.006161+0000","last_clean":"2024-09-11T09:31:48.006161+0000","last_became_active":"2024-09-11T09:31:45.707966+0000","last_became_peered":"2024-09-11T09:31:45.707966+0000","last_unstale":"2024-09-11T09:31:48.006161+0000","last_undegraded":"2024-09-11T09:31:48.006161+0000","last_fullsized":"2024-09-11T09:31:48.006161+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:08.229335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"151'419","reported_seq":4809,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698278+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:47.698278+0000","last_peered":"2024-09-11T09:31:47.698278+0000","last_clean":"2024-09-11T09:31:47.698278+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:47.698278+0000","last_undegraded":"2024-09-11T09:31:47.698278+0000","last_fullsized":"2024-09-11T09:31:47.698278+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7674,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698421+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:47.698421+0000","last_peered":"2024-09-11T09:31:47.698421+0000","last_clean":"2024-09-11T09:31:47.698421+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:47.698421+0000","last_undegraded":"2024-09-11T09:31:47.698421+0000","last_fullsized":"2024-09-11T09:31:47.698421+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607597,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73508,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298332,"statfs":{"total":96636764160,"available":96561491968,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607596,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77664,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294176,"statfs":{"total":96636764160,"available":96557236224,"internally_reserved":0,"allocated":2101248,"data_stored":891630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2138112,"data_stored":1481979,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607597,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29080,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342760,"statfs":{"total":96636764160,"available":96606986240,"internally_reserved":0,"allocated":2097152,"data_stored":891611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607597,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":2117632,"data_stored":896667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:51.850 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:52.141 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:52.154 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T09:31:50.756845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352968,"kb_used_data":12412,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878072,"statfs":{"total":579820584960,"available":579459145728,"internally_reserved":0,"allocated":12709888,"data_stored":6535516,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.052730"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7165,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697661+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:47.697661+0000","last_peered":"2024-09-11T09:31:47.697661+0000","last_clean":"2024-09-11T09:31:47.697661+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:47.697661+0000","last_undegraded":"2024-09-11T09:31:47.697661+0000","last_fullsized":"2024-09-11T09:31:47.697661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"155'718","reported_seq":7978,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006622+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:48.006622+0000","last_peered":"2024-09-11T09:31:48.006622+0000","last_clean":"2024-09-11T09:31:48.006622+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:48.006622+0000","last_undegraded":"2024-09-11T09:31:48.006622+0000","last_fullsized":"2024-09-11T09:31:48.006622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"155'740","reported_seq":14246,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006809+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:48.006809+0000","last_peered":"2024-09-11T09:31:48.006809+0000","last_clean":"2024-09-11T09:31:48.006809+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:48.006809+0000","last_undegraded":"2024-09-11T09:31:48.006809+0000","last_fullsized":"2024-09-11T09:31:48.006809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006282+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:48.006282+0000","last_peered":"2024-09-11T09:31:48.006282+0000","last_clean":"2024-09-11T09:31:48.006282+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:48.006282+0000","last_undegraded":"2024-09-11T09:31:48.006282+0000","last_fullsized":"2024-09-11T09:31:48.006282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"151'697","reported_seq":8424,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005465+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:48.005465+0000","last_peered":"2024-09-11T09:31:48.005465+0000","last_clean":"2024-09-11T09:31:48.005465+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:48.005465+0000","last_undegraded":"2024-09-11T09:31:48.005465+0000","last_fullsized":"2024-09-11T09:31:48.005465+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"151'490","reported_seq":3317,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697747+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:47.697747+0000","last_peered":"2024-09-11T09:31:47.697747+0000","last_clean":"2024-09-11T09:31:47.697747+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:47.697747+0000","last_undegraded":"2024-09-11T09:31:47.697747+0000","last_fullsized":"2024-09-11T09:31:47.697747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10534,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007045+0000","last_change":"2024-09-11T09:31:45.711457+0000","last_active":"2024-09-11T09:31:48.007045+0000","last_peered":"2024-09-11T09:31:48.007045+0000","last_clean":"2024-09-11T09:31:48.007045+0000","last_became_active":"2024-09-11T09:31:45.709608+0000","last_became_peered":"2024-09-11T09:31:45.709608+0000","last_unstale":"2024-09-11T09:31:48.007045+0000","last_undegraded":"2024-09-11T09:31:48.007045+0000","last_fullsized":"2024-09-11T09:31:48.007045+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:31:35.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'602","reported_seq":6294,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698462+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:47.698462+0000","last_peered":"2024-09-11T09:31:47.698462+0000","last_clean":"2024-09-11T09:31:47.698462+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:47.698462+0000","last_undegraded":"2024-09-11T09:31:47.698462+0000","last_fullsized":"2024-09-11T09:31:47.698462+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"151'682","reported_seq":8996,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698307+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:47.698307+0000","last_peered":"2024-09-11T09:31:47.698307+0000","last_clean":"2024-09-11T09:31:47.698307+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:47.698307+0000","last_undegraded":"2024-09-11T09:31:47.698307+0000","last_fullsized":"2024-09-11T09:31:47.698307+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698629+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:47.698629+0000","last_peered":"2024-09-11T09:31:47.698629+0000","last_clean":"2024-09-11T09:31:47.698629+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:47.698629+0000","last_undegraded":"2024-09-11T09:31:47.698629+0000","last_fullsized":"2024-09-11T09:31:47.698629+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005498+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:48.005498+0000","last_peered":"2024-09-11T09:31:48.005498+0000","last_clean":"2024-09-11T09:31:48.005498+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:48.005498+0000","last_undegraded":"2024-09-11T09:31:48.005498+0000","last_fullsized":"2024-09-11T09:31:48.005498+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7488,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698027+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:47.698027+0000","last_peered":"2024-09-11T09:31:47.698027+0000","last_clean":"2024-09-11T09:31:47.698027+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:47.698027+0000","last_undegraded":"2024-09-11T09:31:47.698027+0000","last_fullsized":"2024-09-11T09:31:47.698027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698338+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:47.698338+0000","last_peered":"2024-09-11T09:31:47.698338+0000","last_clean":"2024-09-11T09:31:47.698338+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:47.698338+0000","last_undegraded":"2024-09-11T09:31:47.698338+0000","last_fullsized":"2024-09-11T09:31:47.698338+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7109,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005760+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:48.005760+0000","last_peered":"2024-09-11T09:31:48.005760+0000","last_clean":"2024-09-11T09:31:48.005760+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:48.005760+0000","last_undegraded":"2024-09-11T09:31:48.005760+0000","last_fullsized":"2024-09-11T09:31:48.005760+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":293,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005569+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:48.005569+0000","last_peered":"2024-09-11T09:31:48.005569+0000","last_clean":"2024-09-11T09:31:48.005569+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:48.005569+0000","last_undegraded":"2024-09-11T09:31:48.005569+0000","last_fullsized":"2024-09-11T09:31:48.005569+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5969,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007050+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:48.007050+0000","last_peered":"2024-09-11T09:31:48.007050+0000","last_clean":"2024-09-11T09:31:48.007050+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:48.007050+0000","last_undegraded":"2024-09-11T09:31:48.007050+0000","last_fullsized":"2024-09-11T09:31:48.007050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'990","reported_seq":13953,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006403+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:48.006403+0000","last_peered":"2024-09-11T09:31:48.006403+0000","last_clean":"2024-09-11T09:31:48.006403+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:48.006403+0000","last_undegraded":"2024-09-11T09:31:48.006403+0000","last_fullsized":"2024-09-11T09:31:48.006403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":336,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005987+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:48.005987+0000","last_peered":"2024-09-11T09:31:48.005987+0000","last_clean":"2024-09-11T09:31:48.005987+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:48.005987+0000","last_undegraded":"2024-09-11T09:31:48.005987+0000","last_fullsized":"2024-09-11T09:31:48.005987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005768+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:48.005768+0000","last_peered":"2024-09-11T09:31:48.005768+0000","last_clean":"2024-09-11T09:31:48.005768+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:48.005768+0000","last_undegraded":"2024-09-11T09:31:48.005768+0000","last_fullsized":"2024-09-11T09:31:48.005768+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698244+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:47.698244+0000","last_peered":"2024-09-11T09:31:47.698244+0000","last_clean":"2024-09-11T09:31:47.698244+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:47.698244+0000","last_undegraded":"2024-09-11T09:31:47.698244+0000","last_fullsized":"2024-09-11T09:31:47.698244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":219,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698143+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:47.698143+0000","last_peered":"2024-09-11T09:31:47.698143+0000","last_clean":"2024-09-11T09:31:47.698143+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:47.698143+0000","last_undegraded":"2024-09-11T09:31:47.698143+0000","last_fullsized":"2024-09-11T09:31:47.698143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":6010,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005747+0000","last_change":"2024-09-11T09:31:44.718923+0000","last_active":"2024-09-11T09:31:48.005747+0000","last_peered":"2024-09-11T09:31:48.005747+0000","last_clean":"2024-09-11T09:31:48.005747+0000","last_became_active":"2024-09-11T09:31:44.717222+0000","last_became_peered":"2024-09-11T09:31:44.717222+0000","last_unstale":"2024-09-11T09:31:48.005747+0000","last_undegraded":"2024-09-11T09:31:48.005747+0000","last_fullsized":"2024-09-11T09:31:48.005747+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:42.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":353,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006600+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:48.006600+0000","last_peered":"2024-09-11T09:31:48.006600+0000","last_clean":"2024-09-11T09:31:48.006600+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:48.006600+0000","last_undegraded":"2024-09-11T09:31:48.006600+0000","last_fullsized":"2024-09-11T09:31:48.006600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698175+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:47.698175+0000","last_peered":"2024-09-11T09:31:47.698175+0000","last_clean":"2024-09-11T09:31:47.698175+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:47.698175+0000","last_undegraded":"2024-09-11T09:31:47.698175+0000","last_fullsized":"2024-09-11T09:31:47.698175+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697540+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:47.697540+0000","last_peered":"2024-09-11T09:31:47.697540+0000","last_clean":"2024-09-11T09:31:47.697540+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:47.697540+0000","last_undegraded":"2024-09-11T09:31:47.697540+0000","last_fullsized":"2024-09-11T09:31:47.697540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"155'620","reported_seq":9286,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006280+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:48.006280+0000","last_peered":"2024-09-11T09:31:48.006280+0000","last_clean":"2024-09-11T09:31:48.006280+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:48.006280+0000","last_undegraded":"2024-09-11T09:31:48.006280+0000","last_fullsized":"2024-09-11T09:31:48.006280+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"152'817","reported_seq":8387,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698918+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:47.698918+0000","last_peered":"2024-09-11T09:31:47.698918+0000","last_clean":"2024-09-11T09:31:47.698918+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:47.698918+0000","last_undegraded":"2024-09-11T09:31:47.698918+0000","last_fullsized":"2024-09-11T09:31:47.698918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"149'361","reported_seq":8252,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005580+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:48.005580+0000","last_peered":"2024-09-11T09:31:48.005580+0000","last_clean":"2024-09-11T09:31:48.005580+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:48.005580+0000","last_undegraded":"2024-09-11T09:31:48.005580+0000","last_fullsized":"2024-09-11T09:31:48.005580+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'381","reported_seq":6591,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006368+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:48.006368+0000","last_peered":"2024-09-11T09:31:48.006368+0000","last_clean":"2024-09-11T09:31:48.006368+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:48.006368+0000","last_undegraded":"2024-09-11T09:31:48.006368+0000","last_fullsized":"2024-09-11T09:31:48.006368+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"149'593","reported_seq":6521,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698333+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:47.698333+0000","last_peered":"2024-09-11T09:31:47.698333+0000","last_clean":"2024-09-11T09:31:47.698333+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:47.698333+0000","last_undegraded":"2024-09-11T09:31:47.698333+0000","last_fullsized":"2024-09-11T09:31:47.698333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"155'530","reported_seq":6858,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006381+0000","last_change":"2024-09-11T09:31:44.711140+0000","last_active":"2024-09-11T09:31:48.006381+0000","last_peered":"2024-09-11T09:31:48.006381+0000","last_clean":"2024-09-11T09:31:48.006381+0000","last_became_active":"2024-09-11T09:31:44.709474+0000","last_became_peered":"2024-09-11T09:31:44.709474+0000","last_unstale":"2024-09-11T09:31:48.006381+0000","last_undegraded":"2024-09-11T09:31:48.006381+0000","last_fullsized":"2024-09-11T09:31:48.006381+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:08.396097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"152'459","reported_seq":13012,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006957+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:48.006957+0000","last_peered":"2024-09-11T09:31:48.006957+0000","last_clean":"2024-09-11T09:31:48.006957+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:48.006957+0000","last_undegraded":"2024-09-11T09:31:48.006957+0000","last_fullsized":"2024-09-11T09:31:48.006957+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"151'841","reported_seq":12855,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006161+0000","last_change":"2024-09-11T09:31:45.710037+0000","last_active":"2024-09-11T09:31:48.006161+0000","last_peered":"2024-09-11T09:31:48.006161+0000","last_clean":"2024-09-11T09:31:48.006161+0000","last_became_active":"2024-09-11T09:31:45.707966+0000","last_became_peered":"2024-09-11T09:31:45.707966+0000","last_unstale":"2024-09-11T09:31:48.006161+0000","last_undegraded":"2024-09-11T09:31:48.006161+0000","last_fullsized":"2024-09-11T09:31:48.006161+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:08.229335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"151'419","reported_seq":4809,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698278+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:47.698278+0000","last_peered":"2024-09-11T09:31:47.698278+0000","last_clean":"2024-09-11T09:31:47.698278+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:47.698278+0000","last_undegraded":"2024-09-11T09:31:47.698278+0000","last_fullsized":"2024-09-11T09:31:47.698278+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7674,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698421+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:47.698421+0000","last_peered":"2024-09-11T09:31:47.698421+0000","last_clean":"2024-09-11T09:31:47.698421+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:47.698421+0000","last_undegraded":"2024-09-11T09:31:47.698421+0000","last_fullsized":"2024-09-11T09:31:47.698421+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607597,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73508,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298332,"statfs":{"total":96636764160,"available":96561491968,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607596,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77664,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294176,"statfs":{"total":96636764160,"available":96557236224,"internally_reserved":0,"allocated":2101248,"data_stored":891630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2138112,"data_stored":1481979,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607597,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29080,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342760,"statfs":{"total":96636764160,"available":96606986240,"internally_reserved":0,"allocated":2097152,"data_stored":891611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607597,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":2117632,"data_stored":896667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:52.156 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T09:31:52.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:52.445 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:52.457 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T09:31:50.756845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":352968,"kb_used_data":12412,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878072,"statfs":{"total":579820584960,"available":579459145728,"internally_reserved":0,"allocated":12709888,"data_stored":6535516,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.052730"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5180,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707135+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:45.707135+0000","last_peered":"2024-09-11T09:31:45.707135+0000","last_clean":"2024-09-11T09:31:45.707135+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:45.707135+0000","last_undegraded":"2024-09-11T09:31:45.707135+0000","last_fullsized":"2024-09-11T09:31:45.707135+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1e","version":"155'686","reported_seq":7165,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697661+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:47.697661+0000","last_peered":"2024-09-11T09:31:47.697661+0000","last_clean":"2024-09-11T09:31:47.697661+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:47.697661+0000","last_undegraded":"2024-09-11T09:31:47.697661+0000","last_fullsized":"2024-09-11T09:31:47.697661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"155'718","reported_seq":7978,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006622+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:48.006622+0000","last_peered":"2024-09-11T09:31:48.006622+0000","last_clean":"2024-09-11T09:31:48.006622+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:48.006622+0000","last_undegraded":"2024-09-11T09:31:48.006622+0000","last_fullsized":"2024-09-11T09:31:48.006622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"155'740","reported_seq":14246,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006809+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:48.006809+0000","last_peered":"2024-09-11T09:31:48.006809+0000","last_clean":"2024-09-11T09:31:48.006809+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:48.006809+0000","last_undegraded":"2024-09-11T09:31:48.006809+0000","last_fullsized":"2024-09-11T09:31:48.006809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'817","reported_seq":9862,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706829+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:45.706829+0000","last_peered":"2024-09-11T09:31:45.706829+0000","last_clean":"2024-09-11T09:31:45.706829+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:45.706829+0000","last_undegraded":"2024-09-11T09:31:45.706829+0000","last_fullsized":"2024-09-11T09:31:45.706829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.1a","version":"152'462","reported_seq":9173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006282+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:48.006282+0000","last_peered":"2024-09-11T09:31:48.006282+0000","last_clean":"2024-09-11T09:31:48.006282+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:48.006282+0000","last_undegraded":"2024-09-11T09:31:48.006282+0000","last_fullsized":"2024-09-11T09:31:48.006282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"151'697","reported_seq":8424,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005465+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:48.005465+0000","last_peered":"2024-09-11T09:31:48.005465+0000","last_clean":"2024-09-11T09:31:48.005465+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:48.005465+0000","last_undegraded":"2024-09-11T09:31:48.005465+0000","last_fullsized":"2024-09-11T09:31:48.005465+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"151'490","reported_seq":3317,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697747+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:47.697747+0000","last_peered":"2024-09-11T09:31:47.697747+0000","last_clean":"2024-09-11T09:31:47.697747+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:47.697747+0000","last_undegraded":"2024-09-11T09:31:47.697747+0000","last_fullsized":"2024-09-11T09:31:47.697747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"151'437","reported_seq":6598,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.707020+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:45.707020+0000","last_peered":"2024-09-11T09:31:45.707020+0000","last_clean":"2024-09-11T09:31:45.707020+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:45.707020+0000","last_undegraded":"2024-09-11T09:31:45.707020+0000","last_fullsized":"2024-09-11T09:31:45.707020+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.16","version":"155'1170","reported_seq":10534,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007045+0000","last_change":"2024-09-11T09:31:45.711457+0000","last_active":"2024-09-11T09:31:48.007045+0000","last_peered":"2024-09-11T09:31:48.007045+0000","last_clean":"2024-09-11T09:31:48.007045+0000","last_became_active":"2024-09-11T09:31:45.709608+0000","last_became_peered":"2024-09-11T09:31:45.709608+0000","last_unstale":"2024-09-11T09:31:48.007045+0000","last_undegraded":"2024-09-11T09:31:48.007045+0000","last_fullsized":"2024-09-11T09:31:48.007045+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:31:35.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'602","reported_seq":6294,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698462+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:47.698462+0000","last_peered":"2024-09-11T09:31:47.698462+0000","last_clean":"2024-09-11T09:31:47.698462+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:47.698462+0000","last_undegraded":"2024-09-11T09:31:47.698462+0000","last_fullsized":"2024-09-11T09:31:47.698462+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"151'682","reported_seq":8996,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698307+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:47.698307+0000","last_peered":"2024-09-11T09:31:47.698307+0000","last_clean":"2024-09-11T09:31:47.698307+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:47.698307+0000","last_undegraded":"2024-09-11T09:31:47.698307+0000","last_fullsized":"2024-09-11T09:31:47.698307+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698629+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:47.698629+0000","last_peered":"2024-09-11T09:31:47.698629+0000","last_clean":"2024-09-11T09:31:47.698629+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:47.698629+0000","last_undegraded":"2024-09-11T09:31:47.698629+0000","last_fullsized":"2024-09-11T09:31:47.698629+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3522,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706656+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:45.706656+0000","last_peered":"2024-09-11T09:31:45.706656+0000","last_clean":"2024-09-11T09:31:45.706656+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:45.706656+0000","last_undegraded":"2024-09-11T09:31:45.706656+0000","last_fullsized":"2024-09-11T09:31:45.706656+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005498+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:48.005498+0000","last_peered":"2024-09-11T09:31:48.005498+0000","last_clean":"2024-09-11T09:31:48.005498+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:48.005498+0000","last_undegraded":"2024-09-11T09:31:48.005498+0000","last_fullsized":"2024-09-11T09:31:48.005498+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7488,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698027+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:47.698027+0000","last_peered":"2024-09-11T09:31:47.698027+0000","last_clean":"2024-09-11T09:31:47.698027+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:47.698027+0000","last_undegraded":"2024-09-11T09:31:47.698027+0000","last_fullsized":"2024-09-11T09:31:47.698027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698338+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:47.698338+0000","last_peered":"2024-09-11T09:31:47.698338+0000","last_clean":"2024-09-11T09:31:47.698338+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:47.698338+0000","last_undegraded":"2024-09-11T09:31:47.698338+0000","last_fullsized":"2024-09-11T09:31:47.698338+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7109,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005760+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:48.005760+0000","last_peered":"2024-09-11T09:31:48.005760+0000","last_clean":"2024-09-11T09:31:48.005760+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:48.005760+0000","last_undegraded":"2024-09-11T09:31:48.005760+0000","last_fullsized":"2024-09-11T09:31:48.005760+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":293,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005569+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:48.005569+0000","last_peered":"2024-09-11T09:31:48.005569+0000","last_clean":"2024-09-11T09:31:48.005569+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:48.005569+0000","last_undegraded":"2024-09-11T09:31:48.005569+0000","last_fullsized":"2024-09-11T09:31:48.005569+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5969,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007050+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:48.007050+0000","last_peered":"2024-09-11T09:31:48.007050+0000","last_clean":"2024-09-11T09:31:48.007050+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:48.007050+0000","last_undegraded":"2024-09-11T09:31:48.007050+0000","last_fullsized":"2024-09-11T09:31:48.007050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'990","reported_seq":13953,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006403+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:48.006403+0000","last_peered":"2024-09-11T09:31:48.006403+0000","last_clean":"2024-09-11T09:31:48.006403+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:48.006403+0000","last_undegraded":"2024-09-11T09:31:48.006403+0000","last_fullsized":"2024-09-11T09:31:48.006403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":336,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005987+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:48.005987+0000","last_peered":"2024-09-11T09:31:48.005987+0000","last_clean":"2024-09-11T09:31:48.005987+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:48.005987+0000","last_undegraded":"2024-09-11T09:31:48.005987+0000","last_fullsized":"2024-09-11T09:31:48.005987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005768+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:48.005768+0000","last_peered":"2024-09-11T09:31:48.005768+0000","last_clean":"2024-09-11T09:31:48.005768+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:48.005768+0000","last_undegraded":"2024-09-11T09:31:48.005768+0000","last_fullsized":"2024-09-11T09:31:48.005768+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698244+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:47.698244+0000","last_peered":"2024-09-11T09:31:47.698244+0000","last_clean":"2024-09-11T09:31:47.698244+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:47.698244+0000","last_undegraded":"2024-09-11T09:31:47.698244+0000","last_fullsized":"2024-09-11T09:31:47.698244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10112,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706604+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:45.706604+0000","last_peered":"2024-09-11T09:31:45.706604+0000","last_clean":"2024-09-11T09:31:45.706604+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:45.706604+0000","last_undegraded":"2024-09-11T09:31:45.706604+0000","last_fullsized":"2024-09-11T09:31:45.706604+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"2.0","version":"0'0","reported_seq":219,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698143+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:47.698143+0000","last_peered":"2024-09-11T09:31:47.698143+0000","last_clean":"2024-09-11T09:31:47.698143+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:47.698143+0000","last_undegraded":"2024-09-11T09:31:47.698143+0000","last_fullsized":"2024-09-11T09:31:47.698143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":6010,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005747+0000","last_change":"2024-09-11T09:31:44.718923+0000","last_active":"2024-09-11T09:31:48.005747+0000","last_peered":"2024-09-11T09:31:48.005747+0000","last_clean":"2024-09-11T09:31:48.005747+0000","last_became_active":"2024-09-11T09:31:44.717222+0000","last_became_peered":"2024-09-11T09:31:44.717222+0000","last_unstale":"2024-09-11T09:31:48.005747+0000","last_undegraded":"2024-09-11T09:31:48.005747+0000","last_fullsized":"2024-09-11T09:31:48.005747+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:42.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":353,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006600+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:48.006600+0000","last_peered":"2024-09-11T09:31:48.006600+0000","last_clean":"2024-09-11T09:31:48.006600+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:48.006600+0000","last_undegraded":"2024-09-11T09:31:48.006600+0000","last_fullsized":"2024-09-11T09:31:48.006600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698175+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:47.698175+0000","last_peered":"2024-09-11T09:31:47.698175+0000","last_clean":"2024-09-11T09:31:47.698175+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:47.698175+0000","last_undegraded":"2024-09-11T09:31:47.698175+0000","last_fullsized":"2024-09-11T09:31:47.698175+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697540+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:47.697540+0000","last_peered":"2024-09-11T09:31:47.697540+0000","last_clean":"2024-09-11T09:31:47.697540+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:47.697540+0000","last_undegraded":"2024-09-11T09:31:47.697540+0000","last_fullsized":"2024-09-11T09:31:47.697540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"155'620","reported_seq":9286,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006280+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:48.006280+0000","last_peered":"2024-09-11T09:31:48.006280+0000","last_clean":"2024-09-11T09:31:48.006280+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:48.006280+0000","last_undegraded":"2024-09-11T09:31:48.006280+0000","last_fullsized":"2024-09-11T09:31:48.006280+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"152'817","reported_seq":8387,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698918+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:47.698918+0000","last_peered":"2024-09-11T09:31:47.698918+0000","last_clean":"2024-09-11T09:31:47.698918+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:47.698918+0000","last_undegraded":"2024-09-11T09:31:47.698918+0000","last_fullsized":"2024-09-11T09:31:47.698918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"149'361","reported_seq":8252,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005580+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:48.005580+0000","last_peered":"2024-09-11T09:31:48.005580+0000","last_clean":"2024-09-11T09:31:48.005580+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:48.005580+0000","last_undegraded":"2024-09-11T09:31:48.005580+0000","last_fullsized":"2024-09-11T09:31:48.005580+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'381","reported_seq":6591,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006368+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:48.006368+0000","last_peered":"2024-09-11T09:31:48.006368+0000","last_clean":"2024-09-11T09:31:48.006368+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:48.006368+0000","last_undegraded":"2024-09-11T09:31:48.006368+0000","last_fullsized":"2024-09-11T09:31:48.006368+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"149'593","reported_seq":6521,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698333+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:47.698333+0000","last_peered":"2024-09-11T09:31:47.698333+0000","last_clean":"2024-09-11T09:31:47.698333+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:47.698333+0000","last_undegraded":"2024-09-11T09:31:47.698333+0000","last_fullsized":"2024-09-11T09:31:47.698333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"155'530","reported_seq":6858,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006381+0000","last_change":"2024-09-11T09:31:44.711140+0000","last_active":"2024-09-11T09:31:48.006381+0000","last_peered":"2024-09-11T09:31:48.006381+0000","last_clean":"2024-09-11T09:31:48.006381+0000","last_became_active":"2024-09-11T09:31:44.709474+0000","last_became_peered":"2024-09-11T09:31:44.709474+0000","last_unstale":"2024-09-11T09:31:48.006381+0000","last_undegraded":"2024-09-11T09:31:48.006381+0000","last_fullsized":"2024-09-11T09:31:48.006381+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:08.396097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"152'459","reported_seq":13012,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006957+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:48.006957+0000","last_peered":"2024-09-11T09:31:48.006957+0000","last_clean":"2024-09-11T09:31:48.006957+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:48.006957+0000","last_undegraded":"2024-09-11T09:31:48.006957+0000","last_fullsized":"2024-09-11T09:31:48.006957+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"151'841","reported_seq":12855,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006161+0000","last_change":"2024-09-11T09:31:45.710037+0000","last_active":"2024-09-11T09:31:48.006161+0000","last_peered":"2024-09-11T09:31:48.006161+0000","last_clean":"2024-09-11T09:31:48.006161+0000","last_became_active":"2024-09-11T09:31:45.707966+0000","last_became_peered":"2024-09-11T09:31:45.707966+0000","last_unstale":"2024-09-11T09:31:48.006161+0000","last_undegraded":"2024-09-11T09:31:48.006161+0000","last_fullsized":"2024-09-11T09:31:48.006161+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:08.229335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"151'419","reported_seq":4809,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698278+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:47.698278+0000","last_peered":"2024-09-11T09:31:47.698278+0000","last_clean":"2024-09-11T09:31:47.698278+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:47.698278+0000","last_undegraded":"2024-09-11T09:31:47.698278+0000","last_fullsized":"2024-09-11T09:31:47.698278+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'882","reported_seq":8867,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-11T09:31:45.706693+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:45.706693+0000","last_peered":"2024-09-11T09:31:45.706693+0000","last_clean":"2024-09-11T09:31:45.706693+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:45.706693+0000","last_undegraded":"2024-09-11T09:31:45.706693+0000","last_fullsized":"2024-09-11T09:31:45.706693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"2b"}]},{"pgid":"3.13","version":"155'735","reported_seq":7674,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698421+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:47.698421+0000","last_peered":"2024-09-11T09:31:47.698421+0000","last_clean":"2024-09-11T09:31:47.698421+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:47.698421+0000","last_undegraded":"2024-09-11T09:31:47.698421+0000","last_fullsized":"2024-09-11T09:31:47.698421+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607597,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73508,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298332,"statfs":{"total":96636764160,"available":96561491968,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607596,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77664,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294176,"statfs":{"total":96636764160,"available":96557236224,"internally_reserved":0,"allocated":2101248,"data_stored":891630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2138112,"data_stored":1481979,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607597,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29080,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342760,"statfs":{"total":96636764160,"available":96606986240,"internally_reserved":0,"allocated":2097152,"data_stored":891611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607597,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":2117632,"data_stored":896667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734661,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35392,"kb_used_data":2092,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336448,"statfs":{"total":96636764160,"available":96600522752,"internally_reserved":0,"allocated":2142208,"data_stored":1476981,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:52.459 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T09:31:52.748 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:52.749 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":176,"fsid":"23a8bf95-caba-4fea-a762-61040a1c1b77","created":"2024-09-11T09:28:17.894525+0000","modified":"2024-09-11T09:31:47.694741+0000","last_up_change":"2024-09-11T09:31:31.888533+0000","last_in_change":"2024-09-11T09:31:31.888533+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T09:28:28.765227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-11T09:28:34.774979+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-11T09:28:45.573818+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":153,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"44ce15f5-731e-4d58-9dd7-bd8f186e5783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":11,"last_clean_end":89,"up_from":161,"up_thru":173,"down_at":90,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6801","nonce":2589762321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6803","nonce":2589762321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6807","nonce":2589762321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2589762321},{"type":"v1","addr":"172.21.15.17:6805","nonce":2589762321}]},"public_addr":"172.21.15.17:6801/2589762321","cluster_addr":"172.21.15.17:6803/2589762321","heartbeat_back_addr":"172.21.15.17:6807/2589762321","heartbeat_front_addr":"172.21.15.17:6805/2589762321","state":["exists","up"]},{"osd":1,"uuid":"dba82e38-a74b-4d42-bad4-9b1e013d4da3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6817","nonce":361445038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6819","nonce":361445038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6823","nonce":361445038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":361445038},{"type":"v1","addr":"172.21.15.17:6821","nonce":361445038}]},"public_addr":"172.21.15.17:6817/361445038","cluster_addr":"172.21.15.17:6819/361445038","heartbeat_back_addr":"172.21.15.17:6823/361445038","heartbeat_front_addr":"172.21.15.17:6821/361445038","state":["exists","up"]},{"osd":2,"uuid":"64a33918-9bb2-4d59-a66e-73da15d78cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6809","nonce":211521567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6811","nonce":211521567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6815","nonce":211521567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":211521567},{"type":"v1","addr":"172.21.15.17:6813","nonce":211521567}]},"public_addr":"172.21.15.17:6809/211521567","cluster_addr":"172.21.15.17:6811/211521567","heartbeat_back_addr":"172.21.15.17:6815/211521567","heartbeat_front_addr":"172.21.15.17:6813/211521567","state":["exists","up"]},{"osd":3,"uuid":"a4774bc9-54ea-4272-b6b4-a6dd00c4c930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6801","nonce":2070567113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6803","nonce":2070567113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6807","nonce":2070567113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2070567113},{"type":"v1","addr":"172.21.15.96:6805","nonce":2070567113}]},"public_addr":"172.21.15.96:6801/2070567113","cluster_addr":"172.21.15.96:6803/2070567113","heartbeat_back_addr":"172.21.15.96:6807/2070567113","heartbeat_front_addr":"172.21.15.96:6805/2070567113","state":["exists","up"]},{"osd":4,"uuid":"a3984c6d-7250-4e85-a032-1dc0315642d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6817","nonce":1795828331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6819","nonce":1795828331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6823","nonce":1795828331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":1795828331},{"type":"v1","addr":"172.21.15.96:6821","nonce":1795828331}]},"public_addr":"172.21.15.96:6817/1795828331","cluster_addr":"172.21.15.96:6819/1795828331","heartbeat_back_addr":"172.21.15.96:6823/1795828331","heartbeat_front_addr":"172.21.15.96:6821/1795828331","state":["exists","up"]},{"osd":5,"uuid":"0a40bbb7-9d37-4a1b-b525-86608a3eaaa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6809","nonce":3808455039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6811","nonce":3808455039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6815","nonce":3808455039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3808455039},{"type":"v1","addr":"172.21.15.96:6813","nonce":3808455039}]},"public_addr":"172.21.15.96:6809/3808455039","cluster_addr":"172.21.15.96:6811/3808455039","heartbeat_back_addr":"172.21.15.96:6815/3808455039","heartbeat_front_addr":"172.21.15.96:6813/3808455039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.320025+0000","dead_epoch":90},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.922939+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.876780+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:25.854159+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.081538+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T09:28:26.168719+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.18","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":3,"length":43}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T09:31:53.762 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T09:31:53.762 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T09:31:53.901 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:53.901 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-11T09:31:53.901 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:53.912 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T09:31:54.206 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T09:31:54.219 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T09:31:54.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T09:31:54.344 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:54.345 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T09:31:54.345 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:54.355 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T09:31:54.649 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T09:31:54.662 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T09:31:54.663 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T09:31:54.788 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:54.788 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T09:31:54.788 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:54.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T09:31:55.090 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T09:31:55.103 INFO:tasks.ceph:Scrubbing osd.3 2024-09-11T09:31:55.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-11T09:31:55.229 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:55.229 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T09:31:55.229 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:55.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-11T09:31:55.529 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 3 to deep-scrub 2024-09-11T09:31:55.542 INFO:tasks.ceph:Scrubbing osd.4 2024-09-11T09:31:55.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-11T09:31:55.670 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:55.670 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T09:31:55.670 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:55.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-11T09:31:55.971 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 4 to deep-scrub 2024-09-11T09:31:55.984 INFO:tasks.ceph:Scrubbing osd.5 2024-09-11T09:31:55.984 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-11T09:31:56.111 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-11T09:31:56.111 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T09:31:56.111 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-11T09:31:56.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-11T09:31:56.413 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 5 to deep-scrub 2024-09-11T09:31:56.426 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:31:56.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:31:56.725 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:31:56.736 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T09:31:54.757535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21939,"ondisk_log_size":21939,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":353008,"kb_used_data":12472,"kb_used_omap":52,"kb_used_meta":340363,"kb_avail":565878032,"statfs":{"total":579820584960,"available":579459104768,"internally_reserved":0,"allocated":12771328,"data_stored":6560586,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":348532475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.052290"},"pg_stats":[{"pgid":"3.1f","version":"157'811","reported_seq":5182,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698726+0000","last_change":"2024-09-11T09:31:34.715605+0000","last_active":"2024-09-11T09:31:47.698726+0000","last_peered":"2024-09-11T09:31:47.698726+0000","last_clean":"2024-09-11T09:31:47.698726+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:47.698726+0000","last_undegraded":"2024-09-11T09:31:47.698726+0000","last_fullsized":"2024-09-11T09:31:47.698726+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:59:39.922293+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"155'686","reported_seq":7165,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697661+0000","last_change":"2024-09-11T09:31:36.718885+0000","last_active":"2024-09-11T09:31:47.697661+0000","last_peered":"2024-09-11T09:31:47.697661+0000","last_clean":"2024-09-11T09:31:47.697661+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:47.697661+0000","last_undegraded":"2024-09-11T09:31:47.697661+0000","last_fullsized":"2024-09-11T09:31:47.697661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:14.114787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"155'718","reported_seq":7978,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006622+0000","last_change":"2024-09-11T09:31:20.864302+0000","last_active":"2024-09-11T09:31:48.006622+0000","last_peered":"2024-09-11T09:31:48.006622+0000","last_clean":"2024-09-11T09:31:48.006622+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:48.006622+0000","last_undegraded":"2024-09-11T09:31:48.006622+0000","last_fullsized":"2024-09-11T09:31:48.006622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:19.715541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"155'740","reported_seq":14246,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006809+0000","last_change":"2024-09-11T09:31:20.865673+0000","last_active":"2024-09-11T09:31:48.006809+0000","last_peered":"2024-09-11T09:31:48.006809+0000","last_clean":"2024-09-11T09:31:48.006809+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:48.006809+0000","last_undegraded":"2024-09-11T09:31:48.006809+0000","last_fullsized":"2024-09-11T09:31:48.006809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:06.375229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"165'817","reported_seq":9864,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697929+0000","last_change":"2024-09-11T09:31:35.819957+0000","last_active":"2024-09-11T09:31:47.697929+0000","last_peered":"2024-09-11T09:31:47.697929+0000","last_clean":"2024-09-11T09:31:47.697929+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:47.697929+0000","last_undegraded":"2024-09-11T09:31:47.697929+0000","last_fullsized":"2024-09-11T09:31:47.697929+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:47.950448+0000","scrub_duration":0,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"152'462","reported_seq":9173,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006282+0000","last_change":"2024-09-11T09:31:43.711685+0000","last_active":"2024-09-11T09:31:48.006282+0000","last_peered":"2024-09-11T09:31:48.006282+0000","last_clean":"2024-09-11T09:31:48.006282+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:48.006282+0000","last_undegraded":"2024-09-11T09:31:48.006282+0000","last_fullsized":"2024-09-11T09:31:48.006282+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:06.383531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"151'697","reported_seq":8424,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005465+0000","last_change":"2024-09-11T09:31:43.708084+0000","last_active":"2024-09-11T09:31:48.005465+0000","last_peered":"2024-09-11T09:31:48.005465+0000","last_clean":"2024-09-11T09:31:48.005465+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:31:48.005465+0000","last_undegraded":"2024-09-11T09:31:48.005465+0000","last_fullsized":"2024-09-11T09:31:48.005465+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:25.804050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"151'490","reported_seq":3317,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697747+0000","last_change":"2024-09-11T09:31:20.858364+0000","last_active":"2024-09-11T09:31:47.697747+0000","last_peered":"2024-09-11T09:31:47.697747+0000","last_clean":"2024-09-11T09:31:47.697747+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:47.697747+0000","last_undegraded":"2024-09-11T09:31:47.697747+0000","last_fullsized":"2024-09-11T09:31:47.697747+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:48.173726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"151'437","reported_seq":6600,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698744+0000","last_change":"2024-09-11T09:31:44.708751+0000","last_active":"2024-09-11T09:31:47.698744+0000","last_peered":"2024-09-11T09:31:47.698744+0000","last_clean":"2024-09-11T09:31:47.698744+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:47.698744+0000","last_undegraded":"2024-09-11T09:31:47.698744+0000","last_fullsized":"2024-09-11T09:31:47.698744+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:37.035762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"155'1170","reported_seq":10534,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007045+0000","last_change":"2024-09-11T09:31:45.711457+0000","last_active":"2024-09-11T09:31:48.007045+0000","last_peered":"2024-09-11T09:31:48.007045+0000","last_clean":"2024-09-11T09:31:48.007045+0000","last_became_active":"2024-09-11T09:31:45.709608+0000","last_became_peered":"2024-09-11T09:31:45.709608+0000","last_unstale":"2024-09-11T09:31:48.007045+0000","last_undegraded":"2024-09-11T09:31:48.007045+0000","last_fullsized":"2024-09-11T09:31:48.007045+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:31:35.658395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'602","reported_seq":6294,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698462+0000","last_change":"2024-09-11T09:31:38.721561+0000","last_active":"2024-09-11T09:31:47.698462+0000","last_peered":"2024-09-11T09:31:47.698462+0000","last_clean":"2024-09-11T09:31:47.698462+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:47.698462+0000","last_undegraded":"2024-09-11T09:31:47.698462+0000","last_fullsized":"2024-09-11T09:31:47.698462+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:45.109659+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"151'682","reported_seq":8996,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698307+0000","last_change":"2024-09-11T09:31:42.704581+0000","last_active":"2024-09-11T09:31:47.698307+0000","last_peered":"2024-09-11T09:31:47.698307+0000","last_clean":"2024-09-11T09:31:47.698307+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:31:47.698307+0000","last_undegraded":"2024-09-11T09:31:47.698307+0000","last_fullsized":"2024-09-11T09:31:47.698307+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:43.840215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":103,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698629+0000","last_change":"2024-09-11T09:31:38.725211+0000","last_active":"2024-09-11T09:31:47.698629+0000","last_peered":"2024-09-11T09:31:47.698629+0000","last_clean":"2024-09-11T09:31:47.698629+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:31:47.698629+0000","last_undegraded":"2024-09-11T09:31:47.698629+0000","last_fullsized":"2024-09-11T09:31:47.698629+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:26.527221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"166'445","reported_seq":3524,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698180+0000","last_change":"2024-09-11T09:31:36.814832+0000","last_active":"2024-09-11T09:31:47.698180+0000","last_peered":"2024-09-11T09:31:47.698180+0000","last_clean":"2024-09-11T09:31:47.698180+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:47.698180+0000","last_undegraded":"2024-09-11T09:31:47.698180+0000","last_fullsized":"2024-09-11T09:31:47.698180+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:31.335147+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":36,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005498+0000","last_change":"2024-09-11T09:31:33.718881+0000","last_active":"2024-09-11T09:31:48.005498+0000","last_peered":"2024-09-11T09:31:48.005498+0000","last_clean":"2024-09-11T09:31:48.005498+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:48.005498+0000","last_undegraded":"2024-09-11T09:31:48.005498+0000","last_fullsized":"2024-09-11T09:31:48.005498+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:38.057269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"150'428","reported_seq":7488,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698027+0000","last_change":"2024-09-11T09:31:20.856167+0000","last_active":"2024-09-11T09:31:47.698027+0000","last_peered":"2024-09-11T09:31:47.698027+0000","last_clean":"2024-09-11T09:31:47.698027+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:47.698027+0000","last_undegraded":"2024-09-11T09:31:47.698027+0000","last_fullsized":"2024-09-11T09:31:47.698027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:31.910663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698338+0000","last_change":"2024-09-11T09:31:38.721501+0000","last_active":"2024-09-11T09:31:47.698338+0000","last_peered":"2024-09-11T09:31:47.698338+0000","last_clean":"2024-09-11T09:31:47.698338+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:47.698338+0000","last_undegraded":"2024-09-11T09:31:47.698338+0000","last_fullsized":"2024-09-11T09:31:47.698338+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:38:05.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"151'740","reported_seq":7109,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005760+0000","last_change":"2024-09-11T09:31:34.709805+0000","last_active":"2024-09-11T09:31:48.005760+0000","last_peered":"2024-09-11T09:31:48.005760+0000","last_clean":"2024-09-11T09:31:48.005760+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:48.005760+0000","last_undegraded":"2024-09-11T09:31:48.005760+0000","last_fullsized":"2024-09-11T09:31:48.005760+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:36:33.522004+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":293,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005569+0000","last_change":"2024-09-11T09:31:34.710349+0000","last_active":"2024-09-11T09:31:48.005569+0000","last_peered":"2024-09-11T09:31:48.005569+0000","last_clean":"2024-09-11T09:31:48.005569+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:48.005569+0000","last_undegraded":"2024-09-11T09:31:48.005569+0000","last_fullsized":"2024-09-11T09:31:48.005569+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:27.518758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"155'541","reported_seq":5969,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.007050+0000","last_change":"2024-09-11T09:31:20.865080+0000","last_active":"2024-09-11T09:31:48.007050+0000","last_peered":"2024-09-11T09:31:48.007050+0000","last_clean":"2024-09-11T09:31:48.007050+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:31:48.007050+0000","last_undegraded":"2024-09-11T09:31:48.007050+0000","last_fullsized":"2024-09-11T09:31:48.007050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:35.576413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"165'990","reported_seq":13953,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006403+0000","last_change":"2024-09-11T09:31:35.801042+0000","last_active":"2024-09-11T09:31:48.006403+0000","last_peered":"2024-09-11T09:31:48.006403+0000","last_clean":"2024-09-11T09:31:48.006403+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:48.006403+0000","last_undegraded":"2024-09-11T09:31:48.006403+0000","last_fullsized":"2024-09-11T09:31:48.006403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:14:55.524415+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":336,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005987+0000","last_change":"2024-09-11T09:28:38.467426+0000","last_active":"2024-09-11T09:31:48.005987+0000","last_peered":"2024-09-11T09:31:48.005987+0000","last_clean":"2024-09-11T09:31:48.005987+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:48.005987+0000","last_undegraded":"2024-09-11T09:31:48.005987+0000","last_fullsized":"2024-09-11T09:31:48.005987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:03.892198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"155'1212","reported_seq":11650,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005768+0000","last_change":"2024-09-11T09:31:34.709978+0000","last_active":"2024-09-11T09:31:48.005768+0000","last_peered":"2024-09-11T09:31:48.005768+0000","last_clean":"2024-09-11T09:31:48.005768+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:48.005768+0000","last_undegraded":"2024-09-11T09:31:48.005768+0000","last_fullsized":"2024-09-11T09:31:48.005768+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:12:02.913794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":216,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698244+0000","last_change":"2024-09-11T09:29:51.768638+0000","last_active":"2024-09-11T09:31:47.698244+0000","last_peered":"2024-09-11T09:31:47.698244+0000","last_clean":"2024-09-11T09:31:47.698244+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:47.698244+0000","last_undegraded":"2024-09-11T09:31:47.698244+0000","last_fullsized":"2024-09-11T09:31:47.698244+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:33.284666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"155'493","reported_seq":10114,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697637+0000","last_change":"2024-09-11T09:31:42.703567+0000","last_active":"2024-09-11T09:31:47.697637+0000","last_peered":"2024-09-11T09:31:47.697637+0000","last_clean":"2024-09-11T09:31:47.697637+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:47.697637+0000","last_undegraded":"2024-09-11T09:31:47.697637+0000","last_fullsized":"2024-09-11T09:31:47.697637+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:26.125611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":219,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698143+0000","last_change":"2024-09-11T09:31:38.719275+0000","last_active":"2024-09-11T09:31:47.698143+0000","last_peered":"2024-09-11T09:31:47.698143+0000","last_clean":"2024-09-11T09:31:47.698143+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:47.698143+0000","last_undegraded":"2024-09-11T09:31:47.698143+0000","last_fullsized":"2024-09-11T09:31:47.698143+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:19.949227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'421","reported_seq":6010,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005747+0000","last_change":"2024-09-11T09:31:44.718923+0000","last_active":"2024-09-11T09:31:48.005747+0000","last_peered":"2024-09-11T09:31:48.005747+0000","last_clean":"2024-09-11T09:31:48.005747+0000","last_became_active":"2024-09-11T09:31:44.717222+0000","last_became_peered":"2024-09-11T09:31:44.717222+0000","last_unstale":"2024-09-11T09:31:48.005747+0000","last_undegraded":"2024-09-11T09:31:48.005747+0000","last_fullsized":"2024-09-11T09:31:48.005747+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:42.949616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":353,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006600+0000","last_change":"2024-09-11T09:29:51.771655+0000","last_active":"2024-09-11T09:31:48.006600+0000","last_peered":"2024-09-11T09:31:48.006600+0000","last_clean":"2024-09-11T09:31:48.006600+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:31:48.006600+0000","last_undegraded":"2024-09-11T09:31:48.006600+0000","last_fullsized":"2024-09-11T09:31:48.006600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:35.443489+0000","last_clean_scrub_stamp":"2024-09-11T09:28:35.443489+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:48.878755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":418,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698175+0000","last_change":"2024-09-11T09:31:38.720134+0000","last_active":"2024-09-11T09:31:47.698175+0000","last_peered":"2024-09-11T09:31:47.698175+0000","last_clean":"2024-09-11T09:31:47.698175+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:47.698175+0000","last_undegraded":"2024-09-11T09:31:47.698175+0000","last_fullsized":"2024-09-11T09:31:47.698175+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:29.425028+0000","last_clean_scrub_stamp":"2024-09-11T09:28:29.425028+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:59:22.693460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"152'1599","reported_seq":13780,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697540+0000","last_change":"2024-09-11T09:31:20.858236+0000","last_active":"2024-09-11T09:31:47.697540+0000","last_peered":"2024-09-11T09:31:47.697540+0000","last_clean":"2024-09-11T09:31:47.697540+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:47.697540+0000","last_undegraded":"2024-09-11T09:31:47.697540+0000","last_fullsized":"2024-09-11T09:31:47.697540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":1599,"log_dups_size":0,"ondisk_log_size":1599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.193772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"155'620","reported_seq":9286,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006280+0000","last_change":"2024-09-11T09:31:42.702194+0000","last_active":"2024-09-11T09:31:48.006280+0000","last_peered":"2024-09-11T09:31:48.006280+0000","last_clean":"2024-09-11T09:31:48.006280+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:48.006280+0000","last_undegraded":"2024-09-11T09:31:48.006280+0000","last_fullsized":"2024-09-11T09:31:48.006280+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:40:05.846498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"152'817","reported_seq":8387,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698918+0000","last_change":"2024-09-11T09:31:43.712412+0000","last_active":"2024-09-11T09:31:47.698918+0000","last_peered":"2024-09-11T09:31:47.698918+0000","last_clean":"2024-09-11T09:31:47.698918+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:31:47.698918+0000","last_undegraded":"2024-09-11T09:31:47.698918+0000","last_fullsized":"2024-09-11T09:31:47.698918+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:13.051541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"149'361","reported_seq":8252,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.005580+0000","last_change":"2024-09-11T09:31:35.342214+0000","last_active":"2024-09-11T09:31:48.005580+0000","last_peered":"2024-09-11T09:31:48.005580+0000","last_clean":"2024-09-11T09:31:48.005580+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:48.005580+0000","last_undegraded":"2024-09-11T09:31:48.005580+0000","last_fullsized":"2024-09-11T09:31:48.005580+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:44.364297+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"162'381","reported_seq":6591,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006368+0000","last_change":"2024-09-11T09:31:32.810142+0000","last_active":"2024-09-11T09:31:48.006368+0000","last_peered":"2024-09-11T09:31:48.006368+0000","last_clean":"2024-09-11T09:31:48.006368+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:31:48.006368+0000","last_undegraded":"2024-09-11T09:31:48.006368+0000","last_fullsized":"2024-09-11T09:31:48.006368+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:10:58.934344+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"149'593","reported_seq":6521,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698333+0000","last_change":"2024-09-11T09:31:38.720700+0000","last_active":"2024-09-11T09:31:47.698333+0000","last_peered":"2024-09-11T09:31:47.698333+0000","last_clean":"2024-09-11T09:31:47.698333+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:47.698333+0000","last_undegraded":"2024-09-11T09:31:47.698333+0000","last_fullsized":"2024-09-11T09:31:47.698333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:34:48.918836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"155'530","reported_seq":6858,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006381+0000","last_change":"2024-09-11T09:31:44.711140+0000","last_active":"2024-09-11T09:31:48.006381+0000","last_peered":"2024-09-11T09:31:48.006381+0000","last_clean":"2024-09-11T09:31:48.006381+0000","last_became_active":"2024-09-11T09:31:44.709474+0000","last_became_peered":"2024-09-11T09:31:44.709474+0000","last_unstale":"2024-09-11T09:31:48.006381+0000","last_undegraded":"2024-09-11T09:31:48.006381+0000","last_fullsized":"2024-09-11T09:31:48.006381+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:12:08.396097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"152'459","reported_seq":13012,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006957+0000","last_change":"2024-09-11T09:31:32.708367+0000","last_active":"2024-09-11T09:31:48.006957+0000","last_peered":"2024-09-11T09:31:48.006957+0000","last_clean":"2024-09-11T09:31:48.006957+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:31:48.006957+0000","last_undegraded":"2024-09-11T09:31:48.006957+0000","last_fullsized":"2024-09-11T09:31:48.006957+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:26.086330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"151'841","reported_seq":12855,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:48.006161+0000","last_change":"2024-09-11T09:31:45.710037+0000","last_active":"2024-09-11T09:31:48.006161+0000","last_peered":"2024-09-11T09:31:48.006161+0000","last_clean":"2024-09-11T09:31:48.006161+0000","last_became_active":"2024-09-11T09:31:45.707966+0000","last_became_peered":"2024-09-11T09:31:45.707966+0000","last_unstale":"2024-09-11T09:31:48.006161+0000","last_undegraded":"2024-09-11T09:31:48.006161+0000","last_fullsized":"2024-09-11T09:31:48.006161+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":841,"log_dups_size":0,"ondisk_log_size":841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:18:08.229335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"151'419","reported_seq":4809,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698278+0000","last_change":"2024-09-11T09:31:38.723650+0000","last_active":"2024-09-11T09:31:47.698278+0000","last_peered":"2024-09-11T09:31:47.698278+0000","last_clean":"2024-09-11T09:31:47.698278+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:31:47.698278+0000","last_undegraded":"2024-09-11T09:31:47.698278+0000","last_fullsized":"2024-09-11T09:31:47.698278+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:58.271080+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"158'882","reported_seq":8869,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.697723+0000","last_change":"2024-09-11T09:31:34.725897+0000","last_active":"2024-09-11T09:31:47.697723+0000","last_peered":"2024-09-11T09:31:47.697723+0000","last_clean":"2024-09-11T09:31:47.697723+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:47.697723+0000","last_undegraded":"2024-09-11T09:31:47.697723+0000","last_fullsized":"2024-09-11T09:31:47.697723+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:30.738818+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"155'735","reported_seq":7674,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:47.698421+0000","last_change":"2024-09-11T09:31:38.723354+0000","last_active":"2024-09-11T09:31:47.698421+0000","last_peered":"2024-09-11T09:31:47.698421+0000","last_clean":"2024-09-11T09:31:47.698421+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:31:47.698421+0000","last_undegraded":"2024-09-11T09:31:47.698421+0000","last_fullsized":"2024-09-11T09:31:47.698421+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:28:45.681996+0000","last_clean_scrub_stamp":"2024-09-11T09:28:45.681996+0000","objects_scrubbed":0,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:49:00.598354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":21819,"ondisk_log_size":21819,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607598,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73508,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298332,"statfs":{"total":96636764160,"available":96561491968,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607597,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77668,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294172,"statfs":{"total":96636764160,"available":96557232128,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607597,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59656,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312184,"statfs":{"total":96636764160,"available":96575676416,"internally_reserved":0,"allocated":2150400,"data_stored":1486997,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607598,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29088,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342752,"statfs":{"total":96636764160,"available":96606978048,"internally_reserved":0,"allocated":2109440,"data_stored":896629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607598,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77672,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294168,"statfs":{"total":96636764160,"available":96557228032,"internally_reserved":0,"allocated":2117632,"data_stored":896667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734662,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35416,"kb_used_data":2116,"kb_used_omap":44,"kb_used_meta":33235,"kb_avail":94336424,"statfs":{"total":96636764160,"available":96600498176,"internally_reserved":0,"allocated":2166784,"data_stored":1486997,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":34033184},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:31:56.741 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.741 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.741 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.741 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.741 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.741 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.742 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.743 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-11T09:28:35.443489+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T09:28:29.425028+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=29, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.744 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.745 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-11T09:28:45.681996+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=28, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=9, tm_min=31, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T09:31:56.746 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T09:32:16.746 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T09:32:17.052 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-11T09:32:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-11T09:32:17.064 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-11T09:32:16.762601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":405,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":405,"num_whiteouts":397,"num_read":295212,"num_read_kb":1379374,"num_write":24012,"num_write_kb":433280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8254,"num_bytes_recovered":210205067,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22344,"ondisk_log_size":22344,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":375192,"kb_used_data":12472,"kb_used_omap":52,"kb_used_meta":362571,"kb_avail":565855848,"statfs":{"total":579820584960,"available":579436388352,"internally_reserved":0,"allocated":12771328,"data_stored":6560586,"data_compressed":18388,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":53509,"internal_metadata":371273467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002799"},"pg_stats":[{"pgid":"3.1f","version":"176'823","reported_seq":5203,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:54.614901+0000","last_change":"2024-09-11T09:31:54.614937+0000","last_active":"2024-09-11T09:31:54.614901+0000","last_peered":"2024-09-11T09:31:54.614901+0000","last_clean":"2024-09-11T09:31:54.614901+0000","last_became_active":"2024-09-11T09:31:34.712100+0000","last_became_peered":"2024-09-11T09:31:34.712100+0000","last_unstale":"2024-09-11T09:31:54.614901+0000","last_undegraded":"2024-09-11T09:31:54.614901+0000","last_fullsized":"2024-09-11T09:31:54.614901+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'823","last_scrub_stamp":"2024-09-11T09:31:54.614820+0000","last_deep_scrub":"176'823","last_deep_scrub_stamp":"2024-09-11T09:31:54.614820+0000","last_clean_scrub_stamp":"2024-09-11T09:31:54.614820+0000","objects_scrubbed":12,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:58:05.575689+0000","scrub_duration":7,"objects_trimmed":40,"snaptrim_duration":0.113018249,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6317,"num_read_kb":34753,"num_write":905,"num_write_kb":19895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":26751520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'699","reported_seq":7188,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:59.685461+0000","last_change":"2024-09-11T09:31:59.685505+0000","last_active":"2024-09-11T09:31:59.685461+0000","last_peered":"2024-09-11T09:31:59.685461+0000","last_clean":"2024-09-11T09:31:59.685461+0000","last_became_active":"2024-09-11T09:31:36.716764+0000","last_became_peered":"2024-09-11T09:31:36.716764+0000","last_unstale":"2024-09-11T09:31:59.685461+0000","last_undegraded":"2024-09-11T09:31:59.685461+0000","last_fullsized":"2024-09-11T09:31:59.685461+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'699","last_scrub_stamp":"2024-09-11T09:31:59.685368+0000","last_deep_scrub":"176'699","last_deep_scrub_stamp":"2024-09-11T09:31:59.685368+0000","last_clean_scrub_stamp":"2024-09-11T09:31:59.685368+0000","objects_scrubbed":13,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:17:06.474420+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0014947910000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7325,"num_read_kb":52747,"num_write":715,"num_write_kb":17108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":7851151,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"176'736","reported_seq":8005,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:58.142727+0000","last_change":"2024-09-11T09:31:58.142777+0000","last_active":"2024-09-11T09:31:58.142727+0000","last_peered":"2024-09-11T09:31:58.142727+0000","last_clean":"2024-09-11T09:31:58.142727+0000","last_became_active":"2024-09-11T09:28:46.701616+0000","last_became_peered":"2024-09-11T09:28:46.701616+0000","last_unstale":"2024-09-11T09:31:58.142727+0000","last_undegraded":"2024-09-11T09:31:58.142727+0000","last_fullsized":"2024-09-11T09:31:58.142727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'736","last_scrub_stamp":"2024-09-11T09:31:58.142634+0000","last_deep_scrub":"176'736","last_deep_scrub_stamp":"2024-09-11T09:31:58.142634+0000","last_clean_scrub_stamp":"2024-09-11T09:31:58.142634+0000","objects_scrubbed":18,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:21.551399+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0020413710000000002,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":9439,"num_read_kb":45000,"num_write":854,"num_write_kb":13285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'754","reported_seq":14269,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:59.146848+0000","last_change":"2024-09-11T09:31:59.146909+0000","last_active":"2024-09-11T09:31:59.146848+0000","last_peered":"2024-09-11T09:31:59.146848+0000","last_clean":"2024-09-11T09:31:59.146848+0000","last_became_active":"2024-09-11T09:28:46.702128+0000","last_became_peered":"2024-09-11T09:28:46.702128+0000","last_unstale":"2024-09-11T09:31:59.146848+0000","last_undegraded":"2024-09-11T09:31:59.146848+0000","last_fullsized":"2024-09-11T09:31:59.146848+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'754","last_scrub_stamp":"2024-09-11T09:31:59.146754+0000","last_deep_scrub":"176'754","last_deep_scrub_stamp":"2024-09-11T09:31:59.146754+0000","last_clean_scrub_stamp":"2024-09-11T09:31:59.146754+0000","objects_scrubbed":14,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:27.720133+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001526225,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":15355,"num_read_kb":64161,"num_write":832,"num_write_kb":11691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'825","reported_seq":9881,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:56.629548+0000","last_change":"2024-09-11T09:31:56.629582+0000","last_active":"2024-09-11T09:31:56.629548+0000","last_peered":"2024-09-11T09:31:56.629548+0000","last_clean":"2024-09-11T09:31:56.629548+0000","last_became_active":"2024-09-11T09:31:35.714488+0000","last_became_peered":"2024-09-11T09:31:35.714488+0000","last_unstale":"2024-09-11T09:31:56.629548+0000","last_undegraded":"2024-09-11T09:31:56.629548+0000","last_fullsized":"2024-09-11T09:31:56.629548+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'825","last_scrub_stamp":"2024-09-11T09:31:56.629476+0000","last_deep_scrub":"176'825","last_deep_scrub_stamp":"2024-09-11T09:31:56.629476+0000","last_clean_scrub_stamp":"2024-09-11T09:31:56.629476+0000","objects_scrubbed":8,"log_size":825,"log_dups_size":0,"ondisk_log_size":825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:14:22.152038+0000","scrub_duration":6,"objects_trimmed":40,"snaptrim_duration":0.102297126,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":11724,"num_read_kb":35575,"num_write":1030,"num_write_kb":17494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":63488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"176'477","reported_seq":9197,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:58.567178+0000","last_change":"2024-09-11T09:31:58.567231+0000","last_active":"2024-09-11T09:31:58.567178+0000","last_peered":"2024-09-11T09:31:58.567178+0000","last_clean":"2024-09-11T09:31:58.567178+0000","last_became_active":"2024-09-11T09:31:43.710168+0000","last_became_peered":"2024-09-11T09:31:43.710168+0000","last_unstale":"2024-09-11T09:31:58.567178+0000","last_undegraded":"2024-09-11T09:31:58.567178+0000","last_fullsized":"2024-09-11T09:31:58.567178+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'477","last_scrub_stamp":"2024-09-11T09:31:58.567091+0000","last_deep_scrub":"176'477","last_deep_scrub_stamp":"2024-09-11T09:31:58.567091+0000","last_clean_scrub_stamp":"2024-09-11T09:31:58.567091+0000","objects_scrubbed":15,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:59:15.407756+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0024593750000000002,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":8767,"num_read_kb":42586,"num_write":552,"num_write_kb":14591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":2301107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"176'704","reported_seq":8440,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:00.230907+0000","last_change":"2024-09-11T09:32:00.230983+0000","last_active":"2024-09-11T09:32:00.230907+0000","last_peered":"2024-09-11T09:32:00.230907+0000","last_clean":"2024-09-11T09:32:00.230907+0000","last_became_active":"2024-09-11T09:31:43.706202+0000","last_became_peered":"2024-09-11T09:31:43.706202+0000","last_unstale":"2024-09-11T09:32:00.230907+0000","last_undegraded":"2024-09-11T09:32:00.230907+0000","last_fullsized":"2024-09-11T09:32:00.230907+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'704","last_scrub_stamp":"2024-09-11T09:32:00.230793+0000","last_deep_scrub":"176'704","last_deep_scrub_stamp":"2024-09-11T09:32:00.230793+0000","last_clean_scrub_stamp":"2024-09-11T09:32:00.230793+0000","objects_scrubbed":7,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:48:01.523806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.001523979,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8563,"num_read_kb":59933,"num_write":705,"num_write_kb":29061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":568,"num_bytes_recovered":11246258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'503","reported_seq":3339,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:58.727404+0000","last_change":"2024-09-11T09:31:58.727447+0000","last_active":"2024-09-11T09:31:58.727404+0000","last_peered":"2024-09-11T09:31:58.727404+0000","last_clean":"2024-09-11T09:31:58.727404+0000","last_became_active":"2024-09-11T09:30:31.979281+0000","last_became_peered":"2024-09-11T09:30:31.979281+0000","last_unstale":"2024-09-11T09:31:58.727404+0000","last_undegraded":"2024-09-11T09:31:58.727404+0000","last_fullsized":"2024-09-11T09:31:58.727404+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'503","last_scrub_stamp":"2024-09-11T09:31:58.727313+0000","last_deep_scrub":"176'503","last_deep_scrub_stamp":"2024-09-11T09:31:58.727313+0000","last_clean_scrub_stamp":"2024-09-11T09:31:58.727313+0000","objects_scrubbed":13,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:11:33.736353+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0038058520000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":3818,"num_read_kb":24327,"num_write":485,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":8505464,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"176'444","reported_seq":6616,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:59.582631+0000","last_change":"2024-09-11T09:31:59.582663+0000","last_active":"2024-09-11T09:31:59.582631+0000","last_peered":"2024-09-11T09:31:59.582631+0000","last_clean":"2024-09-11T09:31:59.582631+0000","last_became_active":"2024-09-11T09:31:44.706488+0000","last_became_peered":"2024-09-11T09:31:44.706488+0000","last_unstale":"2024-09-11T09:31:59.582631+0000","last_undegraded":"2024-09-11T09:31:59.582631+0000","last_fullsized":"2024-09-11T09:31:59.582631+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'444","last_scrub_stamp":"2024-09-11T09:31:59.582534+0000","last_deep_scrub":"176'444","last_deep_scrub_stamp":"2024-09-11T09:31:59.582534+0000","last_clean_scrub_stamp":"2024-09-11T09:31:59.582534+0000","objects_scrubbed":7,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:55:37.008635+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.003213605,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5933,"num_read_kb":48085,"num_write":517,"num_write_kb":15440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":8382661,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'1182","reported_seq":10555,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:06.047053+0000","last_change":"2024-09-11T09:32:06.047097+0000","last_active":"2024-09-11T09:32:06.047053+0000","last_peered":"2024-09-11T09:32:06.047053+0000","last_clean":"2024-09-11T09:32:06.047053+0000","last_became_active":"2024-09-11T09:31:45.709608+0000","last_became_peered":"2024-09-11T09:31:45.709608+0000","last_unstale":"2024-09-11T09:32:06.047053+0000","last_undegraded":"2024-09-11T09:32:06.047053+0000","last_fullsized":"2024-09-11T09:32:06.047053+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'1182","last_scrub_stamp":"2024-09-11T09:32:06.046968+0000","last_deep_scrub":"176'1182","last_deep_scrub_stamp":"2024-09-11T09:32:06.046968+0000","last_clean_scrub_stamp":"2024-09-11T09:32:06.046968+0000","objects_scrubbed":12,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:43.376646+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0011503380000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":14902,"num_read_kb":43024,"num_write":1393,"num_write_kb":9686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":581,"num_bytes_recovered":4236786,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'616","reported_seq":6318,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:59.572449+0000","last_change":"2024-09-11T09:31:59.572449+0000","last_active":"2024-09-11T09:31:59.572449+0000","last_peered":"2024-09-11T09:31:59.572449+0000","last_clean":"2024-09-11T09:31:59.572449+0000","last_became_active":"2024-09-11T09:31:38.719743+0000","last_became_peered":"2024-09-11T09:31:38.719743+0000","last_unstale":"2024-09-11T09:31:59.572449+0000","last_undegraded":"2024-09-11T09:31:59.572449+0000","last_fullsized":"2024-09-11T09:31:59.572449+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'616","last_scrub_stamp":"2024-09-11T09:31:59.572282+0000","last_deep_scrub":"176'616","last_deep_scrub_stamp":"2024-09-11T09:31:59.572282+0000","last_clean_scrub_stamp":"2024-09-11T09:31:59.572282+0000","objects_scrubbed":14,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:37.055575+0000","scrub_duration":12,"objects_trimmed":36,"snaptrim_duration":0.097575554999999994,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6565,"num_read_kb":35060,"num_write":550,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":4709026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"176'691","reported_seq":9014,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:03.663841+0000","last_change":"2024-09-11T09:32:03.663896+0000","last_active":"2024-09-11T09:32:03.663841+0000","last_peered":"2024-09-11T09:32:03.663841+0000","last_clean":"2024-09-11T09:32:03.663841+0000","last_became_active":"2024-09-11T09:31:42.702695+0000","last_became_peered":"2024-09-11T09:31:42.702695+0000","last_unstale":"2024-09-11T09:32:03.663841+0000","last_undegraded":"2024-09-11T09:32:03.663841+0000","last_fullsized":"2024-09-11T09:32:03.663841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'691","last_scrub_stamp":"2024-09-11T09:32:03.663752+0000","last_deep_scrub":"176'691","last_deep_scrub_stamp":"2024-09-11T09:32:03.663752+0000","last_clean_scrub_stamp":"2024-09-11T09:32:03.663752+0000","objects_scrubbed":9,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:12:52.310488+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0020556210000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":8627,"num_read_kb":43951,"num_write":767,"num_write_kb":10114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":7081588,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:02.601745+0000","last_change":"2024-09-11T09:32:02.601793+0000","last_active":"2024-09-11T09:32:02.601745+0000","last_peered":"2024-09-11T09:32:02.601745+0000","last_clean":"2024-09-11T09:32:02.601745+0000","last_became_active":"2024-09-11T09:31:38.724918+0000","last_became_peered":"2024-09-11T09:31:38.724918+0000","last_unstale":"2024-09-11T09:32:02.601745+0000","last_undegraded":"2024-09-11T09:32:02.601745+0000","last_fullsized":"2024-09-11T09:32:02.601745+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:32:02.601631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:32:02.601631+0000","last_clean_scrub_stamp":"2024-09-11T09:32:02.601631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:21:26.852320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"176'455","reported_seq":3543,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:57.597056+0000","last_change":"2024-09-11T09:31:57.597105+0000","last_active":"2024-09-11T09:31:57.597056+0000","last_peered":"2024-09-11T09:31:57.597056+0000","last_clean":"2024-09-11T09:31:57.597056+0000","last_became_active":"2024-09-11T09:31:36.718568+0000","last_became_peered":"2024-09-11T09:31:36.718568+0000","last_unstale":"2024-09-11T09:31:57.597056+0000","last_undegraded":"2024-09-11T09:31:57.597056+0000","last_fullsized":"2024-09-11T09:31:57.597056+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'455","last_scrub_stamp":"2024-09-11T09:31:57.596955+0000","last_deep_scrub":"176'455","last_deep_scrub_stamp":"2024-09-11T09:31:57.596955+0000","last_clean_scrub_stamp":"2024-09-11T09:31:57.596955+0000","objects_scrubbed":10,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:13:20.753203+0000","scrub_duration":8,"objects_trimmed":38,"snaptrim_duration":0.094093101999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3002,"num_read_kb":22620,"num_write":445,"num_write_kb":9955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":2168320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":45,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:55.191088+0000","last_change":"2024-09-11T09:31:55.191107+0000","last_active":"2024-09-11T09:31:55.191088+0000","last_peered":"2024-09-11T09:31:55.191088+0000","last_clean":"2024-09-11T09:31:55.191088+0000","last_became_active":"2024-09-11T09:31:33.718491+0000","last_became_peered":"2024-09-11T09:31:33.718491+0000","last_unstale":"2024-09-11T09:31:55.191088+0000","last_undegraded":"2024-09-11T09:31:55.191088+0000","last_fullsized":"2024-09-11T09:31:55.191088+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:31:55.191009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:31:55.191009+0000","last_clean_scrub_stamp":"2024-09-11T09:31:55.191009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:34.634476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000685588,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'442","reported_seq":7512,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:56.693004+0000","last_change":"2024-09-11T09:31:56.693004+0000","last_active":"2024-09-11T09:31:56.693004+0000","last_peered":"2024-09-11T09:31:56.693004+0000","last_clean":"2024-09-11T09:31:56.693004+0000","last_became_active":"2024-09-11T09:29:54.785256+0000","last_became_peered":"2024-09-11T09:29:54.785256+0000","last_unstale":"2024-09-11T09:31:56.693004+0000","last_undegraded":"2024-09-11T09:31:56.693004+0000","last_fullsized":"2024-09-11T09:31:56.693004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'442","last_scrub_stamp":"2024-09-11T09:31:56.692891+0000","last_deep_scrub":"176'442","last_deep_scrub_stamp":"2024-09-11T09:31:56.692891+0000","last_clean_scrub_stamp":"2024-09-11T09:31:56.692891+0000","objects_scrubbed":14,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:29:25.887972+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.002705846,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7206,"num_read_kb":32862,"num_write":385,"num_write_kb":12941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":2862080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":225,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:58.570443+0000","last_change":"2024-09-11T09:31:58.570470+0000","last_active":"2024-09-11T09:31:58.570443+0000","last_peered":"2024-09-11T09:31:58.570443+0000","last_clean":"2024-09-11T09:31:58.570443+0000","last_became_active":"2024-09-11T09:31:38.721072+0000","last_became_peered":"2024-09-11T09:31:38.721072+0000","last_unstale":"2024-09-11T09:31:58.570443+0000","last_undegraded":"2024-09-11T09:31:58.570443+0000","last_fullsized":"2024-09-11T09:31:58.570443+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:31:58.570350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:31:58.570350+0000","last_clean_scrub_stamp":"2024-09-11T09:31:58.570350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:59.945925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"176'754","reported_seq":7132,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:56.227920+0000","last_change":"2024-09-11T09:31:56.227962+0000","last_active":"2024-09-11T09:31:56.227920+0000","last_peered":"2024-09-11T09:31:56.227920+0000","last_clean":"2024-09-11T09:31:56.227920+0000","last_became_active":"2024-09-11T09:31:34.708126+0000","last_became_peered":"2024-09-11T09:31:34.708126+0000","last_unstale":"2024-09-11T09:31:56.227920+0000","last_undegraded":"2024-09-11T09:31:56.227920+0000","last_fullsized":"2024-09-11T09:31:56.227920+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'754","last_scrub_stamp":"2024-09-11T09:31:56.227848+0000","last_deep_scrub":"176'754","last_deep_scrub_stamp":"2024-09-11T09:31:56.227848+0000","last_clean_scrub_stamp":"2024-09-11T09:31:56.227848+0000","objects_scrubbed":14,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:57:37.773167+0000","scrub_duration":9,"objects_trimmed":4,"snaptrim_duration":0.0062546900000000002,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7528,"num_read_kb":35829,"num_write":733,"num_write_kb":17374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":30550616,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":302,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:58.250142+0000","last_change":"2024-09-11T09:31:58.250169+0000","last_active":"2024-09-11T09:31:58.250142+0000","last_peered":"2024-09-11T09:31:58.250142+0000","last_clean":"2024-09-11T09:31:58.250142+0000","last_became_active":"2024-09-11T09:31:34.710083+0000","last_became_peered":"2024-09-11T09:31:34.710083+0000","last_unstale":"2024-09-11T09:31:58.250142+0000","last_undegraded":"2024-09-11T09:31:58.250142+0000","last_fullsized":"2024-09-11T09:31:58.250142+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:31:58.250025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:31:58.250025+0000","last_clean_scrub_stamp":"2024-09-11T09:31:58.250025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:32:41.875845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"176'557","reported_seq":5994,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:01.110355+0000","last_change":"2024-09-11T09:32:01.110406+0000","last_active":"2024-09-11T09:32:01.110355+0000","last_peered":"2024-09-11T09:32:01.110355+0000","last_clean":"2024-09-11T09:32:01.110355+0000","last_became_active":"2024-09-11T09:29:54.792102+0000","last_became_peered":"2024-09-11T09:29:54.792102+0000","last_unstale":"2024-09-11T09:32:01.110355+0000","last_undegraded":"2024-09-11T09:32:01.110355+0000","last_fullsized":"2024-09-11T09:32:01.110355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'557","last_scrub_stamp":"2024-09-11T09:32:01.110270+0000","last_deep_scrub":"176'557","last_deep_scrub_stamp":"2024-09-11T09:32:01.110270+0000","last_clean_scrub_stamp":"2024-09-11T09:32:01.110270+0000","objects_scrubbed":16,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:52:18.284484+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0025886170000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":6199,"num_read_kb":47068,"num_write":509,"num_write_kb":8038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":10890336,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"176'1005","reported_seq":13977,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:56.559982+0000","last_change":"2024-09-11T09:31:56.560054+0000","last_active":"2024-09-11T09:31:56.559982+0000","last_peered":"2024-09-11T09:31:56.559982+0000","last_clean":"2024-09-11T09:31:56.559982+0000","last_became_active":"2024-09-11T09:31:35.718175+0000","last_became_peered":"2024-09-11T09:31:35.718175+0000","last_unstale":"2024-09-11T09:31:56.559982+0000","last_undegraded":"2024-09-11T09:31:56.559982+0000","last_fullsized":"2024-09-11T09:31:56.559982+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'1005","last_scrub_stamp":"2024-09-11T09:31:56.559882+0000","last_deep_scrub":"176'1005","last_deep_scrub_stamp":"2024-09-11T09:31:56.559882+0000","last_clean_scrub_stamp":"2024-09-11T09:31:56.559882+0000","objects_scrubbed":15,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:48:36.445373+0000","scrub_duration":13,"objects_trimmed":46,"snaptrim_duration":0.081962469999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":15501,"num_read_kb":43414,"num_write":894,"num_write_kb":8639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":240640,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"176'3","reported_seq":346,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:57.175910+0000","last_change":"2024-09-11T09:31:57.175948+0000","last_active":"2024-09-11T09:31:57.175910+0000","last_peered":"2024-09-11T09:31:57.175910+0000","last_clean":"2024-09-11T09:31:57.175910+0000","last_became_active":"2024-09-11T09:28:36.457338+0000","last_became_peered":"2024-09-11T09:28:36.457338+0000","last_unstale":"2024-09-11T09:31:57.175910+0000","last_undegraded":"2024-09-11T09:31:57.175910+0000","last_fullsized":"2024-09-11T09:31:57.175910+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'3","last_scrub_stamp":"2024-09-11T09:31:57.175805+0000","last_deep_scrub":"176'3","last_deep_scrub_stamp":"2024-09-11T09:31:57.175805+0000","last_clean_scrub_stamp":"2024-09-11T09:31:57.175805+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:08:16.551535+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0010816140000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"176'1225","reported_seq":11672,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:57.219584+0000","last_change":"2024-09-11T09:31:57.219632+0000","last_active":"2024-09-11T09:31:57.219584+0000","last_peered":"2024-09-11T09:31:57.219584+0000","last_clean":"2024-09-11T09:31:57.219584+0000","last_became_active":"2024-09-11T09:31:34.708077+0000","last_became_peered":"2024-09-11T09:31:34.708077+0000","last_unstale":"2024-09-11T09:31:57.219584+0000","last_undegraded":"2024-09-11T09:31:57.219584+0000","last_fullsized":"2024-09-11T09:31:57.219584+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'1225","last_scrub_stamp":"2024-09-11T09:31:57.219467+0000","last_deep_scrub":"176'1225","last_deep_scrub_stamp":"2024-09-11T09:31:57.219467+0000","last_clean_scrub_stamp":"2024-09-11T09:31:57.219467+0000","objects_scrubbed":13,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:42:14.178836+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00421412,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":14597,"num_read_kb":59245,"num_write":1761,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":2232425,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":226,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:55.670088+0000","last_change":"2024-09-11T09:31:55.670088+0000","last_active":"2024-09-11T09:31:55.670088+0000","last_peered":"2024-09-11T09:31:55.670088+0000","last_clean":"2024-09-11T09:31:55.670088+0000","last_became_active":"2024-09-11T09:29:51.768253+0000","last_became_peered":"2024-09-11T09:29:51.768253+0000","last_unstale":"2024-09-11T09:31:55.670088+0000","last_undegraded":"2024-09-11T09:31:55.670088+0000","last_fullsized":"2024-09-11T09:31:55.670088+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:31:55.669980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:31:55.669980+0000","last_clean_scrub_stamp":"2024-09-11T09:31:55.669980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:13:27.604983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"176'504","reported_seq":10134,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:58.631818+0000","last_change":"2024-09-11T09:31:58.631895+0000","last_active":"2024-09-11T09:31:58.631818+0000","last_peered":"2024-09-11T09:31:58.631818+0000","last_clean":"2024-09-11T09:31:58.631818+0000","last_became_active":"2024-09-11T09:31:42.701186+0000","last_became_peered":"2024-09-11T09:31:42.701186+0000","last_unstale":"2024-09-11T09:31:58.631818+0000","last_undegraded":"2024-09-11T09:31:58.631818+0000","last_fullsized":"2024-09-11T09:31:58.631818+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'504","last_scrub_stamp":"2024-09-11T09:31:58.631707+0000","last_deep_scrub":"176'504","last_deep_scrub_stamp":"2024-09-11T09:31:58.631707+0000","last_clean_scrub_stamp":"2024-09-11T09:31:58.631707+0000","objects_scrubbed":11,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:52.087344+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0037846830000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":11208,"num_read_kb":68932,"num_write":373,"num_write_kb":7236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":5456547,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":228,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:55.586078+0000","last_change":"2024-09-11T09:31:55.586107+0000","last_active":"2024-09-11T09:31:55.586078+0000","last_peered":"2024-09-11T09:31:55.586078+0000","last_clean":"2024-09-11T09:31:55.586078+0000","last_became_active":"2024-09-11T09:31:38.718056+0000","last_became_peered":"2024-09-11T09:31:38.718056+0000","last_unstale":"2024-09-11T09:31:55.586078+0000","last_undegraded":"2024-09-11T09:31:55.586078+0000","last_fullsized":"2024-09-11T09:31:55.586078+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T09:31:55.585983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T09:31:55.585983+0000","last_clean_scrub_stamp":"2024-09-11T09:31:55.585983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:19.998366+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"176'442","reported_seq":6040,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:01.199073+0000","last_change":"2024-09-11T09:32:01.199167+0000","last_active":"2024-09-11T09:32:01.199073+0000","last_peered":"2024-09-11T09:32:01.199073+0000","last_clean":"2024-09-11T09:32:01.199073+0000","last_became_active":"2024-09-11T09:31:44.717222+0000","last_became_peered":"2024-09-11T09:31:44.717222+0000","last_unstale":"2024-09-11T09:32:01.199073+0000","last_undegraded":"2024-09-11T09:32:01.199073+0000","last_fullsized":"2024-09-11T09:32:01.199073+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'442","last_scrub_stamp":"2024-09-11T09:32:01.198971+0000","last_deep_scrub":"176'442","last_deep_scrub_stamp":"2024-09-11T09:32:01.198971+0000","last_clean_scrub_stamp":"2024-09-11T09:32:01.198971+0000","objects_scrubbed":21,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:07:33.564193+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0021345420000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":4243,"num_read_kb":41266,"num_write":413,"num_write_kb":21342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":520,"num_bytes_recovered":2596933,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"176'2","reported_seq":363,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:00.115945+0000","last_change":"2024-09-11T09:32:00.115976+0000","last_active":"2024-09-11T09:32:00.115945+0000","last_peered":"2024-09-11T09:32:00.115945+0000","last_clean":"2024-09-11T09:32:00.115945+0000","last_became_active":"2024-09-11T09:29:51.769166+0000","last_became_peered":"2024-09-11T09:29:51.769166+0000","last_unstale":"2024-09-11T09:32:00.115945+0000","last_undegraded":"2024-09-11T09:32:00.115945+0000","last_fullsized":"2024-09-11T09:32:00.115945+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-11T09:32:00.115848+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-11T09:32:00.115848+0000","last_clean_scrub_stamp":"2024-09-11T09:32:00.115848+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:09:04.535895+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00048552900000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"176'119","reported_seq":429,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:56.590788+0000","last_change":"2024-09-11T09:31:56.590820+0000","last_active":"2024-09-11T09:31:56.590788+0000","last_peered":"2024-09-11T09:31:56.590788+0000","last_clean":"2024-09-11T09:31:56.590788+0000","last_became_active":"2024-09-11T09:31:38.719737+0000","last_became_peered":"2024-09-11T09:31:38.719737+0000","last_unstale":"2024-09-11T09:31:56.590788+0000","last_undegraded":"2024-09-11T09:31:56.590788+0000","last_fullsized":"2024-09-11T09:31:56.590788+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'119","last_scrub_stamp":"2024-09-11T09:31:56.590709+0000","last_deep_scrub":"176'119","last_deep_scrub_stamp":"2024-09-11T09:31:56.590709+0000","last_clean_scrub_stamp":"2024-09-11T09:31:56.590709+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:25:40.095762+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"176'1617","reported_seq":13808,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:57.719194+0000","last_change":"2024-09-11T09:31:57.719194+0000","last_active":"2024-09-11T09:31:57.719194+0000","last_peered":"2024-09-11T09:31:57.719194+0000","last_clean":"2024-09-11T09:31:57.719194+0000","last_became_active":"2024-09-11T09:29:55.787652+0000","last_became_peered":"2024-09-11T09:29:55.787652+0000","last_unstale":"2024-09-11T09:31:57.719194+0000","last_undegraded":"2024-09-11T09:31:57.719194+0000","last_fullsized":"2024-09-11T09:31:57.719194+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'1617","last_scrub_stamp":"2024-09-11T09:31:57.718987+0000","last_deep_scrub":"176'1617","last_deep_scrub_stamp":"2024-09-11T09:31:57.718987+0000","last_clean_scrub_stamp":"2024-09-11T09:31:57.718987+0000","objects_scrubbed":18,"log_size":1617,"log_dups_size":0,"ondisk_log_size":1617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:55.562183+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0043212229999999999,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":18509,"num_read_kb":67048,"num_write":2182,"num_write_kb":15216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":5171566,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"176'629","reported_seq":9304,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:57.580504+0000","last_change":"2024-09-11T09:31:57.580581+0000","last_active":"2024-09-11T09:31:57.580504+0000","last_peered":"2024-09-11T09:31:57.580504+0000","last_clean":"2024-09-11T09:31:57.580504+0000","last_became_active":"2024-09-11T09:31:42.700892+0000","last_became_peered":"2024-09-11T09:31:42.700892+0000","last_unstale":"2024-09-11T09:31:57.580504+0000","last_undegraded":"2024-09-11T09:31:57.580504+0000","last_fullsized":"2024-09-11T09:31:57.580504+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'629","last_scrub_stamp":"2024-09-11T09:31:57.580377+0000","last_deep_scrub":"176'629","last_deep_scrub_stamp":"2024-09-11T09:31:57.580377+0000","last_clean_scrub_stamp":"2024-09-11T09:31:57.580377+0000","objects_scrubbed":9,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:07:05.204670+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0026811449999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":10598,"num_read_kb":46242,"num_write":635,"num_write_kb":13085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":5301550,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'822","reported_seq":8401,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:04.684057+0000","last_change":"2024-09-11T09:32:04.684093+0000","last_active":"2024-09-11T09:32:04.684057+0000","last_peered":"2024-09-11T09:32:04.684057+0000","last_clean":"2024-09-11T09:32:04.684057+0000","last_became_active":"2024-09-11T09:31:43.710600+0000","last_became_peered":"2024-09-11T09:31:43.710600+0000","last_unstale":"2024-09-11T09:32:04.684057+0000","last_undegraded":"2024-09-11T09:32:04.684057+0000","last_fullsized":"2024-09-11T09:32:04.684057+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'822","last_scrub_stamp":"2024-09-11T09:32:04.683961+0000","last_deep_scrub":"176'822","last_deep_scrub_stamp":"2024-09-11T09:32:04.683961+0000","last_clean_scrub_stamp":"2024-09-11T09:32:04.683961+0000","objects_scrubbed":5,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:09.273363+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0020662639999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":4,"num_read":8271,"num_read_kb":44875,"num_write":1137,"num_write_kb":9249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":7796845,"num_keys_recovered":51,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"176'370","reported_seq":8270,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:59.222695+0000","last_change":"2024-09-11T09:31:59.222743+0000","last_active":"2024-09-11T09:31:59.222695+0000","last_peered":"2024-09-11T09:31:59.222695+0000","last_clean":"2024-09-11T09:31:59.222695+0000","last_became_active":"2024-09-11T09:31:34.708326+0000","last_became_peered":"2024-09-11T09:31:34.708326+0000","last_unstale":"2024-09-11T09:31:59.222695+0000","last_undegraded":"2024-09-11T09:31:59.222695+0000","last_fullsized":"2024-09-11T09:31:59.222695+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'370","last_scrub_stamp":"2024-09-11T09:31:59.222603+0000","last_deep_scrub":"176'370","last_deep_scrub_stamp":"2024-09-11T09:31:59.222603+0000","last_clean_scrub_stamp":"2024-09-11T09:31:59.222603+0000","objects_scrubbed":9,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:59:31.634279+0000","scrub_duration":7,"objects_trimmed":1,"snaptrim_duration":0.006950861,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7592,"num_read_kb":35873,"num_write":343,"num_write_kb":5883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3105878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"176'396","reported_seq":6615,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:03.123490+0000","last_change":"2024-09-11T09:32:03.123535+0000","last_active":"2024-09-11T09:32:03.123490+0000","last_peered":"2024-09-11T09:32:03.123490+0000","last_clean":"2024-09-11T09:32:03.123490+0000","last_became_active":"2024-09-11T09:31:32.713762+0000","last_became_peered":"2024-09-11T09:31:32.713762+0000","last_unstale":"2024-09-11T09:32:03.123490+0000","last_undegraded":"2024-09-11T09:32:03.123490+0000","last_fullsized":"2024-09-11T09:32:03.123490+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'396","last_scrub_stamp":"2024-09-11T09:32:03.123408+0000","last_deep_scrub":"176'396","last_deep_scrub_stamp":"2024-09-11T09:32:03.123408+0000","last_clean_scrub_stamp":"2024-09-11T09:32:03.123408+0000","objects_scrubbed":15,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:42:55.756363+0000","scrub_duration":10,"objects_trimmed":28,"snaptrim_duration":0.059918122999999997,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":6567,"num_read_kb":28492,"num_write":331,"num_write_kb":9718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'614","reported_seq":6551,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:57.587139+0000","last_change":"2024-09-11T09:31:57.587230+0000","last_active":"2024-09-11T09:31:57.587139+0000","last_peered":"2024-09-11T09:31:57.587139+0000","last_clean":"2024-09-11T09:31:57.587139+0000","last_became_active":"2024-09-11T09:31:38.717039+0000","last_became_peered":"2024-09-11T09:31:38.717039+0000","last_unstale":"2024-09-11T09:31:57.587139+0000","last_undegraded":"2024-09-11T09:31:57.587139+0000","last_fullsized":"2024-09-11T09:31:57.587139+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'614","last_scrub_stamp":"2024-09-11T09:31:57.587040+0000","last_deep_scrub":"176'614","last_deep_scrub_stamp":"2024-09-11T09:31:57.587040+0000","last_clean_scrub_stamp":"2024-09-11T09:31:57.587040+0000","objects_scrubbed":21,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:53:57.150108+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.001449061,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":6179,"num_read_kb":46774,"num_write":517,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":9044992,"num_keys_recovered":47,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"176'538","reported_seq":6875,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:04.092732+0000","last_change":"2024-09-11T09:32:04.092775+0000","last_active":"2024-09-11T09:32:04.092732+0000","last_peered":"2024-09-11T09:32:04.092732+0000","last_clean":"2024-09-11T09:32:04.092732+0000","last_became_active":"2024-09-11T09:31:44.709474+0000","last_became_peered":"2024-09-11T09:31:44.709474+0000","last_unstale":"2024-09-11T09:32:04.092732+0000","last_undegraded":"2024-09-11T09:32:04.092732+0000","last_fullsized":"2024-09-11T09:32:04.092732+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'538","last_scrub_stamp":"2024-09-11T09:32:04.092645+0000","last_deep_scrub":"176'538","last_deep_scrub_stamp":"2024-09-11T09:32:04.092645+0000","last_clean_scrub_stamp":"2024-09-11T09:32:04.092645+0000","objects_scrubbed":8,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:26:50.030201+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001611183,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":8349,"num_read_kb":38673,"num_write":447,"num_write_kb":10613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":3554320,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"176'470","reported_seq":13032,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:02.078954+0000","last_change":"2024-09-11T09:32:02.079022+0000","last_active":"2024-09-11T09:32:02.078954+0000","last_peered":"2024-09-11T09:32:02.078954+0000","last_clean":"2024-09-11T09:32:02.078954+0000","last_became_active":"2024-09-11T09:31:32.706791+0000","last_became_peered":"2024-09-11T09:31:32.706791+0000","last_unstale":"2024-09-11T09:32:02.078954+0000","last_undegraded":"2024-09-11T09:32:02.078954+0000","last_fullsized":"2024-09-11T09:32:02.078954+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'470","last_scrub_stamp":"2024-09-11T09:32:02.078841+0000","last_deep_scrub":"176'470","last_deep_scrub_stamp":"2024-09-11T09:32:02.078841+0000","last_clean_scrub_stamp":"2024-09-11T09:32:02.078841+0000","objects_scrubbed":11,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:43.924752+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0017728119999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":12971,"num_read_kb":56623,"num_write":479,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":8773230,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"176'849","reported_seq":12872,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:05.086348+0000","last_change":"2024-09-11T09:32:05.086422+0000","last_active":"2024-09-11T09:32:05.086348+0000","last_peered":"2024-09-11T09:32:05.086348+0000","last_clean":"2024-09-11T09:32:05.086348+0000","last_became_active":"2024-09-11T09:31:45.707966+0000","last_became_peered":"2024-09-11T09:31:45.707966+0000","last_unstale":"2024-09-11T09:32:05.086348+0000","last_undegraded":"2024-09-11T09:32:05.086348+0000","last_fullsized":"2024-09-11T09:32:05.086348+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'849","last_scrub_stamp":"2024-09-11T09:32:05.086233+0000","last_deep_scrub":"176'849","last_deep_scrub_stamp":"2024-09-11T09:32:05.086233+0000","last_clean_scrub_stamp":"2024-09-11T09:32:05.086233+0000","objects_scrubbed":8,"log_size":849,"log_dups_size":0,"ondisk_log_size":849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:28.130383+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0036184609999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":14030,"num_read_kb":54878,"num_write":950,"num_write_kb":13403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":5930076,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"176'430","reported_seq":4830,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:01.610333+0000","last_change":"2024-09-11T09:32:01.610333+0000","last_active":"2024-09-11T09:32:01.610333+0000","last_peered":"2024-09-11T09:32:01.610333+0000","last_clean":"2024-09-11T09:32:01.610333+0000","last_became_active":"2024-09-11T09:31:38.722118+0000","last_became_peered":"2024-09-11T09:31:38.722118+0000","last_unstale":"2024-09-11T09:32:01.610333+0000","last_undegraded":"2024-09-11T09:32:01.610333+0000","last_fullsized":"2024-09-11T09:32:01.610333+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'430","last_scrub_stamp":"2024-09-11T09:32:01.610206+0000","last_deep_scrub":"176'430","last_deep_scrub_stamp":"2024-09-11T09:32:01.610206+0000","last_clean_scrub_stamp":"2024-09-11T09:32:01.610206+0000","objects_scrubbed":11,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:18:07.562139+0000","scrub_duration":14,"objects_trimmed":2,"snaptrim_duration":0.00075586499999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":3814,"num_read_kb":26436,"num_write":412,"num_write_kb":22367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":11027456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"176'899","reported_seq":8895,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:31:55.636836+0000","last_change":"2024-09-11T09:31:55.636924+0000","last_active":"2024-09-11T09:31:55.636836+0000","last_peered":"2024-09-11T09:31:55.636836+0000","last_clean":"2024-09-11T09:31:55.636836+0000","last_became_active":"2024-09-11T09:31:34.723270+0000","last_became_peered":"2024-09-11T09:31:34.723270+0000","last_unstale":"2024-09-11T09:31:55.636836+0000","last_undegraded":"2024-09-11T09:31:55.636836+0000","last_fullsized":"2024-09-11T09:31:55.636836+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'899","last_scrub_stamp":"2024-09-11T09:31:55.636727+0000","last_deep_scrub":"176'899","last_deep_scrub_stamp":"2024-09-11T09:31:55.636727+0000","last_clean_scrub_stamp":"2024-09-11T09:31:55.636727+0000","objects_scrubbed":17,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:13.336340+0000","scrub_duration":11,"objects_trimmed":38,"snaptrim_duration":0.069094934999999996,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":12517,"num_read_kb":31807,"num_write":795,"num_write_kb":5930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":9418914,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'748","reported_seq":7696,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-11T09:32:00.566671+0000","last_change":"2024-09-11T09:32:00.566721+0000","last_active":"2024-09-11T09:32:00.566671+0000","last_peered":"2024-09-11T09:32:00.566671+0000","last_clean":"2024-09-11T09:32:00.566671+0000","last_became_active":"2024-09-11T09:31:38.720017+0000","last_became_peered":"2024-09-11T09:31:38.720017+0000","last_unstale":"2024-09-11T09:32:00.566671+0000","last_undegraded":"2024-09-11T09:32:00.566671+0000","last_fullsized":"2024-09-11T09:32:00.566671+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'748","last_scrub_stamp":"2024-09-11T09:32:00.566578+0000","last_deep_scrub":"176'748","last_deep_scrub_stamp":"2024-09-11T09:32:00.566578+0000","last_clean_scrub_stamp":"2024-09-11T09:32:00.566578+0000","objects_scrubbed":13,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:46:06.560086+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0021791990000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":8880,"num_read_kb":21115,"num_write":805,"num_write_kb":9948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":1173998,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":401,"num_object_clones":0,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":401,"num_whiteouts":397,"num_read":295096,"num_read_kb":1379274,"num_write":23851,"num_write_kb":430862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8246,"num_bytes_recovered":208433963,"num_keys_recovered":516,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},"log_size":22220,"ondisk_log_size":22220,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28101,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607602,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73504,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298336,"statfs":{"total":96636764160,"available":96561496064,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607601,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77664,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294176,"statfs":{"total":96636764160,"available":96557236224,"internally_reserved":0,"allocated":2113536,"data_stored":896648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607601,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2150400,"data_stored":1486997,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607602,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47132,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324708,"statfs":{"total":96636764160,"available":96588500992,"internally_reserved":0,"allocated":2109440,"data_stored":896629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607602,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77668,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294172,"statfs":{"total":96636764160,"available":96557232128,"internally_reserved":0,"allocated":2117632,"data_stored":896667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":161,"seq":691489734667,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39572,"kb_used_data":2116,"kb_used_omap":44,"kb_used_meta":37395,"kb_avail":94332268,"statfs":{"total":96636764160,"available":96596242432,"internally_reserved":0,"allocated":2166784,"data_stored":1486997,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":45536,"internal_metadata":38293024},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8675,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9713,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17404,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T09:32:17.067 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T09:32:17.374 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T09:32:17.374 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T09:32:17.374 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T09:32:17.374 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.409 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T09:32:17.410 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T09:32:17.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.484 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T09:32:17.484 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T09:32:17.484 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.557 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T09:32:17.557 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-11T09:32:17.557 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.596 INFO:tasks.ceph.osd.3:Stopped 2024-09-11T09:32:17.596 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-11T09:32:17.596 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.671 INFO:tasks.ceph.osd.4:Stopped 2024-09-11T09:32:17.671 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-11T09:32:17.671 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.759 INFO:tasks.ceph.osd.5:Stopped 2024-09-11T09:32:17.759 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T09:32:17.759 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T09:32:17.759 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.788 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T09:32:17.788 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T09:32:17.788 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T09:32:17.788 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.800 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T09:32:17.800 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T09:32:17.800 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.852 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T09:32:17.852 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T09:32:17.852 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T09:32:17.864 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T09:32:17.865 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T09:32:17.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-11T09:32:17.927 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:32:17.928 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T09:32:18.100 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:32:18.100 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T09:32:18.332 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:32:18.332 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T09:32:18.415 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:32:18.415 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-11T09:32:18.630 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:32:18.631 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-11T09:32:18.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:32:18.717 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-11T09:32:18.782 INFO:tasks.ceph:Archiving mon data... 2024-09-11T09:32:18.783 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/data/mon.a.tgz 2024-09-11T09:32:18.783 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-11T09:32:18.802 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.MkFvxGfji2 2024-09-11T09:32:18.802 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.MkFvxGfji2 2024-09-11T09:32:18.957 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.MkFvxGfji2 2024-09-11T09:32:19.044 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.MkFvxGfji2 is 351KB 2024-09-11T09:32:19.115 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.MkFvxGfji2 2024-09-11T09:32:19.131 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/data/mon.c.tgz 2024-09-11T09:32:19.131 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-11T09:32:19.186 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.7kgsyc7Ndo 2024-09-11T09:32:19.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.7kgsyc7Ndo 2024-09-11T09:32:19.354 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.7kgsyc7Ndo 2024-09-11T09:32:19.439 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.7kgsyc7Ndo is 470KB 2024-09-11T09:32:19.536 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.7kgsyc7Ndo 2024-09-11T09:32:19.552 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/data/mon.b.tgz 2024-09-11T09:32:19.552 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-11T09:32:19.571 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.luD3rAKozN 2024-09-11T09:32:19.572 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.luD3rAKozN 2024-09-11T09:32:19.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.luD3rAKozN 2024-09-11T09:32:19.838 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.luD3rAKozN is 470KB 2024-09-11T09:32:19.934 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.luD3rAKozN 2024-09-11T09:32:19.949 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T09:32:19.949 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T09:32:19.952 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T09:32:19.990 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T09:32:20.042 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T09:32:20.067 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T09:32:20.085 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T09:32:20.085 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T09:32:20.087 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi017/crash 2024-09-11T09:32:20.087 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T09:32:20.113 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi096/crash 2024-09-11T09:32:20.114 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T09:32:20.138 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi107/crash 2024-09-11T09:32:20.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T09:32:20.194 INFO:tasks.ceph:Compressing logs... 2024-09-11T09:32:20.194 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T09:32:20.197 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T09:32:20.199 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T09:32:20.220 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T09:32:20.245 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31492.log 2024-09-11T09:32:20.245 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31532.log 2024-09-11T09:32:20.245 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31492.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-11T09:32:20.245 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31492.log.gz 2024-09-11T09:32:20.245 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31532.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31532.log.gz 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31572.log 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31436.log 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31572.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31572.log.gz 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31436.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31436.log.gz 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T09:32:20.246 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31612.log 2024-09-11T09:32:20.247 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31396.log 2024-09-11T09:32:20.247 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31612.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T09:32:20.247 INFO:teuthology.orchestra.run.smithi096.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31612.log.gz 2024-09-11T09:32:20.247 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-client.admin.31396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31396.log.gz 2024-09-11T09:32:20.247 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-11T09:32:20.247 INFO:teuthology.orchestra.run.smithi096.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T09:32:20.305 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.3.log: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T09:32:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29700.log 2024-09-11T09:32:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27121.log 2024-09-11T09:32:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27493.log 2024-09-11T09:32:20.374 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29700.log.gz 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29174.log 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27121.log.gz 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28552.log 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27493.log.gz 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28938.log 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29174.log.gz 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28972.log 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.28552.log.gz 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29989.log 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.28938.log.gz 2024-09-11T09:32:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27529.log 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.28972.log.gz 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28098.log 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.29989.log.gz 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27669.log 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27529.log.gz 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27992.log 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr: 94.6%/var/log/ceph/ceph-client.admin.27669.log: -- replaced with /var/log/ceph/ceph-client.admin.28098.log.gz 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29033.log 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27669.log.gz 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28719.log 2024-09-11T09:32:20.376 INFO:teuthology.orchestra.run.smithi107.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.27992.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27459.log 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29033.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29594.log 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28719.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27319.log 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27459.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27468.log 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29594.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29849.log 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27319.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29042.log 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.27468.log.gz 2024-09-11T09:32:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 57.0% -- replaced with /var/log/ceph/ceph-client.admin.29849.log.gz 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27633.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.29042.log.gz 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28728.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29210.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27633.log.gz 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29253.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28728.log.gz 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28448.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29210.log.gz 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr:gzip/var/log/ceph/ceph-client.admin.29253.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28999.log 2024-09-11T09:32:20.378 INFO:teuthology.orchestra.run.smithi107.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.29253.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28859.log 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.28448.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27222.log 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28999.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28168.log 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28859.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.27088.log 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.27222.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27739.log 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.admin.28168.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.0.27088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27703.log 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.27088.log.gz 2024-09-11T09:32:20.379 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29743.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27739.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28159.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27703.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28132.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.29743.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29673.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28159.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30120.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.28132.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27432.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.29673.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29664.log 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30120.log.gz 2024-09-11T09:32:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28649.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.27432.log.gz 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28263.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29664.log.gz 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28518.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28649.log.gz 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28412.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28263.log.gz 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29499.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.28518.log.gz 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29639.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 57.3% -- replaced with /var/log/ceph/ceph-client.admin.28412.log.gz 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27389.log 2024-09-11T09:32:20.381 INFO:teuthology.orchestra.run.smithi107.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29499.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28193.log 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.29639.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28062.log 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27389.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29560.log 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28193.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28753.log 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.28062.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30023.log 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29560.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29490.log 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28753.log.gz 2024-09-11T09:32:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29883.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.30023.log.gz 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28692.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29490.log.gz 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29420.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.29883.log.gz 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27642.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.28692.log.gz 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28832.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29420.log.gz 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28789.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 57.6% -- replaced with /var/log/ceph/ceph-client.admin.27642.log.gz 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:gzip/var/log/ceph/ceph-client.admin.28832.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29314.log 2024-09-11T09:32:20.383 INFO:teuthology.orchestra.run.smithi107.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.28832.log.gz 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28342.log 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28789.log.gz 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28202.log 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29314.log.gz 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28378.log 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.28342.log.gz 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28333.log 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr: 35.3% -- replaced with /var/log/ceph/ceph-client.admin.28202.log.gz 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27809.log 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.28378.log.gz 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27748.log 2024-09-11T09:32:20.384 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28333.log.gz 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29813.log 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27809.log.gz 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29103.log 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.27748.log.gz 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27572.log 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.29813.log.gz 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30093.log 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29103.log.gz 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30190.log 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.27572.log.gz 2024-09-11T09:32:20.385 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29874.log 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr: 52.1% -- replaced with /var/log/ceph/ceph-client.admin.30093.log.gz 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30190.log.gz 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29112.log 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29874.log.gz 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29919.log 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27888.log 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr: 78.1% -- replaced with /var/log/ceph/ceph-client.admin.29112.log.gz 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27983.log 2024-09-11T09:32:20.386 INFO:teuthology.orchestra.run.smithi107.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.29919.log.gz 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29219.log 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27983.log: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.27888.log.gz 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29323.log 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27983.log.gz 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28622.log 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.29219.log.gz 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29280.log 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.29323.log.gz 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29569.log 2024-09-11T09:32:20.387 INFO:teuthology.orchestra.run.smithi107.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.28622.log.gz 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29840.log 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29280.log.gz 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29709.log 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.29569.log.gz 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28299.log 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29840.log.gz 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30059.log 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr: 73.5% -- replaced with /var/log/ceph/ceph-client.admin.29709.log.gz 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27563.log 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28299.log.gz 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29244.log 2024-09-11T09:32:20.388 INFO:teuthology.orchestra.run.smithi107.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.30059.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28902.log 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27563.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27353.log 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29244.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27362.log 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr: 25.9% -- replaced with /var/log/ceph/ceph-client.admin.28902.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30014.log 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27353.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28823.log 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.27362.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29734.log 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30014.log.gz 2024-09-11T09:32:20.389 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27949.log 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28823.log.gz 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29630.log 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29734.log.gz 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29289.log 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27949.log.gz 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27913.log 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29630.log.gz 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29770.log 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.29289.log.gz 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28089.log 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27913.log.gz 2024-09-11T09:32:20.390 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27773.log 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29770.log.gz 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28369.log 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28089.log.gz 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27130.log 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27773.log.gz 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28439.log 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28369.log.gz 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29953.log 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr: 56.3% -- replaced with /var/log/ceph/ceph-client.admin.27130.log.gz 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29944.log 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28439.log.gz 2024-09-11T09:32:20.391 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27879.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 35.5% -- replaced with /var/log/ceph/ceph-client.admin.29953.log.gz 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28123.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29944.log.gz 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27502.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27879.log.gz 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29429.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28123.log.gz 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29454.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.27502.log.gz 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30154.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.29429.log.gz 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28683.log 2024-09-11T09:32:20.392 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29454.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27257.log 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30154.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29524.log 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28683.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30199.log 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.27257.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29183.log 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29524.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30129.log 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.30199.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28019.log 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.29183.log.gz 2024-09-11T09:32:20.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27782.log 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.30129.log.gz 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28868.log 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28019.log.gz 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28272.log 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.27782.log.gz 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27712.log 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.28868.log.gz 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29008.log 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr: 52.8% -- replaced with /var/log/ceph/ceph-client.admin.28272.log.gz 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28963.log 2024-09-11T09:32:20.394 INFO:teuthology.orchestra.run.smithi107.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.27712.log.gz 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28588.log 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.29008.log.gz 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29463.log 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28963.log.gz 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29384.log 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.28588.log.gz 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28308.log 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.29463.log.gz 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29804.log 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29384.log.gz 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29533.log 2024-09-11T09:32:20.395 INFO:teuthology.orchestra.run.smithi107.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.28308.log.gz 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28238.log 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29804.log.gz 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29350.log 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.29533.log.gz 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28509.log 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.28238.log.gz 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30163.log 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29350.log.gz 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28473.log 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28509.log.gz 2024-09-11T09:32:20.396 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28613.log 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr: 57.2% -- replaced with /var/log/ceph/ceph-client.admin.30163.log.gz 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28482.log 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28473.log.gz 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28613.log.gz 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29393.log 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr: 52.1% -- replaced with /var/log/ceph/ceph-client.admin.28482.log.gz 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29779.log 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28893.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gz 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29140.log 2024-09-11T09:32:20.397 INFO:teuthology.orchestra.run.smithi107.stderr: 53.7% -- replaced with /var/log/ceph/ceph-client.admin.29393.log.gz 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29980.log 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.29779.log.gz 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29140.log: gzip -5 --verbose 20.0% -- -- replaced with /var/log/ceph/ceph-client.admin.29140.log.gz /var/log/ceph/ceph-client.admin.28543.log 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27248.log 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr: 20.6% -- replaced with /var/log/ceph/ceph-client.admin.29980.log.gz 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28403.log 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28543.log.gz 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28798.log 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27248.log.gz 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27852.log 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28403.log.gz 2024-09-11T09:32:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28762.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.28798.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29359.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.27852.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30084.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.28762.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27398.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.29359.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29149.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30084.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29078.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.admin.27398.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30050.log 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29149.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.29149.log.gz 2024-09-11T09:32:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27599.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.29078.log.gz 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.30050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27678.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30050.log.gz 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27958.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27599.log.gz 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27282.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27678.log: 52.4% -- replaced with /var/log/ceph/ceph-client.admin.27678.log.gz 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29603.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.27958.log.gz 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27213.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27282.log.gz 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-11T09:32:20.400 INFO:teuthology.orchestra.run.smithi107.stderr: 77.8% -- replaced with /var/log/ceph/ceph-client.admin.29603.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27818.log 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27213.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27423.log 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28053.log 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.27818.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28658.log 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27423.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28028.log 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28053.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27922.log 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.admin.28658.log.gz 2024-09-11T09:32:20.401 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27608.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28229.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 75.9% -- replaced with /var/log/ceph/ceph-client.admin.27922.log.gz 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.admin.28028.log.gz 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28579.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.27608.log.gz 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27328.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28229.log.gz 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.28579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27291.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28579.log.gz 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29910.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.27328.log.gz 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27538.log 2024-09-11T09:32:20.402 INFO:teuthology.orchestra.run.smithi107.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.27291.log.gz 2024-09-11T09:32:20.403 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.29910.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29910.log.gz 2024-09-11T09:32:20.403 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.27538.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.27538.log.gz 2024-09-11T09:32:20.403 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-11T09:32:20.403 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m0.063s 2024-09-11T09:32:20.403 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m0.061s 2024-09-11T09:32:20.403 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.092s 2024-09-11T09:32:20.404 INFO:teuthology.orchestra.run.smithi096.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T09:32:20.449 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39900.log 2024-09-11T09:32:20.449 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58786.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.39900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46047.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39900.log.gz 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35002.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58786.log.gz 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56407.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46047.log.gz 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54507.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35002.log.gz 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60453.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56407.log.gz 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52441.log 2024-09-11T09:32:20.450 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54507.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60413.log 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60453.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35479.log 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52441.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60413.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.58607.log -- replaced with /var/log/ceph/ceph-client.admin.60413.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52207.log 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35479.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51081.log 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58607.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35227.log 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52207.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51081.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51081.log.gz 2024-09-11T09:32:20.451 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34788.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35227.log.gz 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56365.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38760.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34788.log.gz 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56365.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.56365.log.gz /var/log/ceph/ceph-client.admin.58649.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38598.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38760.log.gz 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51528.log 2024-09-11T09:32:20.452 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58649.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38598.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51528.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51316.log 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58439.log 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49088.log 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51316.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50236.log 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58439.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49088.log.gz 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-09-11T09:32:20.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59699.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50236.log.gz 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58977.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59825.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59699.log.gz 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44991.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58977.log.gz 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59825.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59825.log.gz 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52044.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44991.log.gz 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55345.log 2024-09-11T09:32:20.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38165.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55345.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45033.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35802.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38165.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36052.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45033.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35802.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56110.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36052.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41663.log 2024-09-11T09:32:20.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56110.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.47587.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56110.log.gz 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41663.log.gz 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38935.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/objectstore_tool.47587.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48559.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38935.log.gz 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39720.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43994.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/objectstore_tool.48559.log: /var/log/ceph/ceph-client.admin.39720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59489.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39720.log.gz 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43994.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43994.log.gz -5 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38405.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37622.log 2024-09-11T09:32:20.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59489.log.gz 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54959.log 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38405.log.gz 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52851.log 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37622.log.gz 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54679.log 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54959.log.gz 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44315.log 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52851.log.gz 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60203.log 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35228.log 2024-09-11T09:32:20.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44315.log.gz 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55263.log 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60203.log.gz 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52767.log 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35228.log.gz 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59333.log 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55263.log.gz 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50804.log 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52767.log.gz 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42200.log 2024-09-11T09:32:20.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59333.log.gz 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31970.log 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50804.log.gz 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.47587.log.gz 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42200.log.gz 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50652.log 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31970.log.gz 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46547.log 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-09-11T09:32:20.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58103.log 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50652.log.gz 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53887.log 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46547.log.gz 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58103.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.58103.log.gz -- 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.43395.log 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53887.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46614.log 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.48559.log.gz 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53887.log.gz 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz /var/log/ceph/ceph-client.admin.58565.log 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/objectstore_tool.46614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45863.log 2024-09-11T09:32:20.460 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39780.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58565.log.gz 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45863.log.gz 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50917.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54639.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39780.log.gz 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50917.log.gz 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52632.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54639.log.gz 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gzgzip 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59657.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41428.log 2024-09-11T09:32:20.461 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52632.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58789.log 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59657.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52809.log 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41428.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38345.log 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58789.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52809.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38995.log 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38345.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz 2024-09-11T09:32:20.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38995.log.gz 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35221.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gz 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50783.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz --verbose 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.59275.log 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.46614.log.gz 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58790.log 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35884.log 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59275.log.gz 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58790.log.gz 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58397.log 2024-09-11T09:32:20.463 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39465.log 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35884.log.gz 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58397.log.gz 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60119.log 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56194.log 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39465.log.gz 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60119.log.gz 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59993.log 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55842.log 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56194.log.gz 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59993.log.gz 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53319.log 2024-09-11T09:32:20.464 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41121.log 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55842.log.gz 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53319.log.gz 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42803.log 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34926.log 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41121.log.gz 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42803.log.gz 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55030.log 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41026.log 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34926.log.gz 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55030.log.gz 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36405.log 2024-09-11T09:32:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.41026.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47168.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41026.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38285.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36405.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.47168.log: --verbose -- /var/log/ceph/ceph-client.admin.59215.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47168.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38285.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38285.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43436.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58523.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59215.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51700.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43436.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59003.log 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58523.log.gz 2024-09-11T09:32:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53421.log 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51700.log.gz 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59003.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 0.0% 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59003.log.gz 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58145.log 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53421.log.gz 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58145.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58145.log.gz 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58999.log 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39251.log 2024-09-11T09:32:20.467 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34870.log 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58999.log.gz 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36795.log 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39251.log.gz 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43670.log 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34870.log.gz 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36795.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42749.log 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36795.log.gz 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log 2024-09-11T09:32:20.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43670.log.gz 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42749.log.gz 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37387.log 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44867.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.52299.log 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41542.log 2024-09-11T09:32:20.469 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37387.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52299.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58355.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52532.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41542.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40626.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58355.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43200.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52532.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51121.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40626.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49493.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43200.log.gz 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50979.log 2024-09-11T09:32:20.470 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51121.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49493.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59447.log 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59531.log 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50979.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38545.log 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59447.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58271.log 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59531.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55416.log 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38545.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53532.log 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58271.log.gz 2024-09-11T09:32:20.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54020.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55416.log.gz 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53532.log.gz 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53655.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54020.log.gz 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53655.log.gz 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37480.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59334.log 2024-09-11T09:32:20.472 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37480.log.gz 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40884.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59334.log.gz 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35632.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40884.log.gz 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36642.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35632.log.gz 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35473.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36642.log.gz 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-09-11T09:32:20.473 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35473.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46272.log 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55751.log 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46272.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36178.log 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58313.log 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55751.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59615.log 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36178.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58313.log.gz 2024-09-11T09:32:20.474 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40482.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59615.log.gz 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58056.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44590.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60287.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40482.log.gz 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58014.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58056.log.gz 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39190.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60287.log.gz 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58014.log.gz 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44805.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37172.log 2024-09-11T09:32:20.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39190.log.gz 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44805.log.gz 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45170.log 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39660.log 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55944.log 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45170.log.gz 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51214.log 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39660.log.gz 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60035.log 2024-09-11T09:32:20.476 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55944.log.gz 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53867.log 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51214.log.gz 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58187.log 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60035.log.gz 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53867.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27595.log 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53867.log.gz 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59783.log 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58187.log.gz 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41141.log 2024-09-11T09:32:20.477 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.tmp-client.admin.27595.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27595.log.gz 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40389.log 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59783.log.gz 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41141.log.gz 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59951.log 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58229.log 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40389.log.gz 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52633.log 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59951.log.gz 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58229.log.gz 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54799.log 2024-09-11T09:32:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.52633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52633.log.gz 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58973.log 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32276.log 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54799.log.gz 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35044.log 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58973.log.gz 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35701.log 0.0% 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32276.log.gz 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35044.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35044.log.gz 2024-09-11T09:32:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35701.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35701.log.gz 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50491.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58791.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50491.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50491.log.gz 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58785.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58791.log.gz 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51680.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45075.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58785.log.gz 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36220.log 2024-09-11T09:32:20.480 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51680.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57046.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45075.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58787.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36220.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54719.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57046.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59867.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58787.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44060.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54719.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59867.log.gz 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40684.log 2024-09-11T09:32:20.481 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44060.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40287.log 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36010.log 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40684.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40287.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40287.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36010.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56833.log 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.56833.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56833.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.54759.log 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-09-11T09:32:20.482 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55112.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36917.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54759.log.gz 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55112.log.gz 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53208.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36917.log: /var/log/ceph/ceph-client.admin.45649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59335.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36917.log.gz 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53208.log.gz 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59335.log: -- /var/log/ceph/ceph-client.admin.54080.log 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59335.log.gz/var/log/ceph/ceph-client.admin.42074.log: 2024-09-11T09:32:20.483 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60245.log 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38111.log 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54080.log: /var/log/ceph/ceph-client.admin.60245.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45117.log 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54080.log.gz 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60245.log.gz 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38111.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38111.log.gz 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53572.log 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45117.log.gz 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39940.log 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53572.log.gz 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41326.log 2024-09-11T09:32:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39940.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39940.log.gz 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42032.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41326.log: 26.7% -- replaced with /var/log/ceph/ceph-client.admin.41326.log.gz 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51973.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42032.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42032.log.gz 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/objectstore_tool.49531.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51973.log.gz 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43711.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/objectstore_tool.49531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46365.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43711.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37837.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46365.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46365.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37837.log.gz 2024-09-11T09:32:20.485 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56068.log 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40696.log 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56068.log.gz 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42158.log 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40696.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38070.log 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32199.log 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42158.log.gz/var/log/ceph/ceph-client.admin.38070.log: 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38070.log.gz 2024-09-11T09:32:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43945.log 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32199.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.54213.log 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32199.log.gz 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr: 94.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42333.log 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/objectstore_tool.49531.log.gz 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43945.log.gz 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54213.log.gz 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59741.log 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42333.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42333.log.gz 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56026.log 2024-09-11T09:32:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59741.log.gz 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44929.log 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56026.log.gz 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51427.log 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44929.log.gz 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36094.log 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51427.log.gz 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41817.log 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37233.log: /var/log/ceph/ceph-client.admin.36094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48100.log 2024-09-11T09:32:20.488 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36094.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41817.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35475.log 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.48100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58481.log 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48100.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35475.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58481.log: /var/log/ceph/ceph-client.admin.52992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58481.log.gz 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56710.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35226.log 2024-09-11T09:32:20.489 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59909.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.35226.log: -- /var/log/ceph/ceph-client.admin.46906.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35226.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59909.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55578.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46906.log: -- /var/log/ceph/ceph-client.admin.35086.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46906.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55578.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60329.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54839.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35086.log: /var/log/ceph/ceph-client.admin.60329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35225.log 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35086.log.gz 2024-09-11T09:32:20.490 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60329.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54839.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54839.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35225.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35225.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59046.log 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35811.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59046.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36970.log 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56619.log 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59195.log 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36970.log.gz 2024-09-11T09:32:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56619.log: -- /var/log/ceph/ceph-client.admin.54315.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56619.log.gz 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59195.log.gz 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38873.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54315.log: -- /var/log/ceph/ceph-client.admin.44691.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54315.log.gz 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38873.log.gz 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35400.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43353.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44691.log.gz 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35400.log.gz 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56578.log 2024-09-11T09:32:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43353.log: gzip/var/log/ceph/ceph-client.admin.38224.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.55984.log 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43353.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56578.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40174.log 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55984.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55984.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45464.log 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40174.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59573.log 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54426.log 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45464.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60077.log 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59573.log.gz 2024-09-11T09:32:20.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56944.log 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54426.log: /var/log/ceph/ceph-client.admin.60077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44457.log 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54426.log.gz 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60077.log.gz 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56944.log.gz 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55518.log 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44457.log: -- /var/log/ceph/ceph-client.admin.46016.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44457.log.gz 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55518.log.gz 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59047.log 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45393.log 2024-09-11T09:32:20.494 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46016.log.gz 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.59047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60161.log 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59047.log.gz 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35399.log 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.45393.log: /var/log/ceph/ceph-client.admin.60161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39518.log 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60161.log.gz 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45393.log.gz 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35399.log.gz 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35652.log 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35224.log 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39518.log: /var/log/ceph/ceph-client.admin.35652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-09-11T09:32:20.495 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39518.log.gz 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35652.log.gz 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37795.log 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35224.log.gz 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56152.log 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42965.log: /var/log/ceph/ceph-client.admin.37795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36282.log 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37795.log.gz 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56152.log.gz 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60371.log 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34951.log 2024-09-11T09:32:20.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36282.log: /var/log/ceph/ceph-client.admin.60371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gz 2024-09-11T09:32:20.497 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60371.log.gz 2024-09-11T09:32:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34951.log.gz 2024-09-11T09:32:20.497 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T09:32:20.590 INFO:teuthology.orchestra.run.smithi017.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T09:32:22.891 INFO:teuthology.orchestra.run.smithi017.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T09:32:24.467 INFO:teuthology.orchestra.run.smithi096.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-11T09:32:24.534 INFO:teuthology.orchestra.run.smithi017.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T09:32:25.287 INFO:teuthology.orchestra.run.smithi096.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-11T09:32:26.359 INFO:teuthology.orchestra.run.smithi017.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T09:32:26.361 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-11T09:32:26.361 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m6.151s 2024-09-11T09:32:26.361 INFO:teuthology.orchestra.run.smithi017.stderr:user 0m13.147s 2024-09-11T09:32:26.361 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m0.618s 2024-09-11T09:32:28.037 INFO:teuthology.orchestra.run.smithi096.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-11T09:32:28.040 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-11T09:32:28.040 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m7.829s 2024-09-11T09:32:28.040 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m16.666s 2024-09-11T09:32:28.040 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.680s 2024-09-11T09:32:28.040 INFO:tasks.ceph:Archiving logs... 2024-09-11T09:32:28.041 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi017/log 2024-09-11T09:32:28.042 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T09:32:29.856 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi096/log 2024-09-11T09:32:29.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T09:32:32.012 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi107/log 2024-09-11T09:32:32.013 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T09:32:32.163 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T09:32:32.182 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T09:32:32.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T09:32:32.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T09:32:32.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T09:32:32.289 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T09:32:32.298 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T09:32:32.299 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:32:32.302 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:32:32.304 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T09:32:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:32:32.319 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:32:32.322 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-11T09:32:32.322 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:32:32.322 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2024-09-11T09:32:32.322 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 37 -2829us[-2828us] +/- 45ms 2024-09-11T09:32:32.322 INFO:teuthology.orchestra.run.smithi017.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 +1977us[+1979us] +/- 55ms 2024-09-11T09:32:32.322 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +1747us[+1747us] +/- 70ms 2024-09-11T09:32:32.323 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:32:32.325 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:32:32.325 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-11T09:32:32.325 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 35 -2821us[-2816us] +/- 45ms 2024-09-11T09:32:32.325 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 +1992us[+1998us] +/- 55ms 2024-09-11T09:32:32.325 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +1756us[+1762us] +/- 70ms 2024-09-11T09:32:32.325 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:32:32.326 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T09:32:32.326 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-11T09:32:32.326 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 -2817us[-2812us] +/- 45ms 2024-09-11T09:32:32.326 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 +2000us[+2005us] +/- 55ms 2024-09-11T09:32:32.327 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +1771us[+1776us] +/- 70ms 2024-09-11T09:32:32.327 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T09:32:32.328 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T09:32:32.337 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T09:32:32.337 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T09:32:32.346 DEBUG:teuthology.orchestra.run.smithi017:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T09:32:32.451 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T09:32:32.530 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T09:32:32.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T09:32:32.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:32:32.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T09:32:32.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:32:32.660 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T09:32:32.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:32:32.692 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T09:32:32.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T09:32:32.721 INFO:teuthology.task.internal:Duration was 657.591908 seconds 2024-09-11T09:32:32.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T09:32:32.729 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T09:32:32.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T09:32:32.732 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T09:32:32.734 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T09:32:32.772 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:32:32.774 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:32:32.775 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T09:32:33.068 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T09:32:33.068 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-09-11T09:32:33.068 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T09:32:33.093 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-11T09:32:33.093 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T09:32:33.132 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-11T09:32:33.132 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T09:32:33.162 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T09:32:33.162 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T09:32:33.165 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T09:32:33.175 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T09:32:33.229 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T09:32:33.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T09:32:33.235 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T09:32:33.243 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T09:32:33.376 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T09:32:33.386 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T09:32:33.386 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T09:32:33.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T09:32:33.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T09:32:33.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T09:32:33.476 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:32:33.478 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:32:33.481 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:32:33.500 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-09-11T09:32:33.501 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-11T09:32:33.526 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-11T09:32:33.551 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:32:33.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:32:33.576 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:32:33.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:32:33.591 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T09:32:33.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T09:32:33.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T09:32:33.617 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T09:32:33.618 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi017 2024-09-11T09:32:33.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T09:32:33.655 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi096 2024-09-11T09:32:33.655 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T09:32:33.689 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_21:16:03-rbd-squid-distro-default-smithi/7900713/remote/smithi107 2024-09-11T09:32:33.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T09:32:33.729 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T09:32:33.729 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T09:32:33.732 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T09:32:33.734 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T09:32:33.776 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T09:32:33.788 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T09:32:33.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T09:32:33.797 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T09:32:33.797 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T09:32:33.800 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T09:32:33.802 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T09:32:33.813 INFO:teuthology.orchestra.run.smithi017.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 09:32 /home/ubuntu/cephtest 2024-09-11T09:32:33.814 INFO:teuthology.orchestra.run.smithi096.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 09:32 /home/ubuntu/cephtest 2024-09-11T09:32:33.831 INFO:teuthology.orchestra.run.smithi107.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 09:32 /home/ubuntu/cephtest 2024-09-11T09:32:33.832 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T09:32:33.843 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T09:32:33.866 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 657.5919079780579 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-11T09:32:33.866 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T09:32:33.964 INFO:teuthology.run:pass